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PREFACE 



In June, 1972, the Research FVmd of the Board of Regents 
of the Hrtiversity^^ of^fevr York, through^the DivislT^^^^ 

for Handicapped Children of the State Education Department was 
awarded a grant to explore the c<Hnpetency based hypothesis and its 
implications for' teacher training particularly with regard to pro- 
viding alternative training strategies and models for the certifi- 
catipn of teachers (The CBTE Project. in Special Education), The 
original objectives of the project were to support CBTE research 
and development - underpinnings in -that context- An action -of the 
Efoaird of Regents in September, 1972 gave great visibility and 
immediate relevancy to the work of the CBTE project. In September, 
1972, the Board of Regents endlDrsed a competence based, field - 
centered approach to the professional preparation of educational 
personnel, effectively mandatiiig that college and university teacher 
preparation facilities active]Ly pursue a competence based approach 
in the prdfessionai training- and educating of prospective public 
school personnel, h - ' ' 

j," ' ' ' * 

New York State defities a competence based 4)rogram of pre- 
paration as one which provides acceptable evidence of program * 
formulation through the collaborative efforts of a consortium of 
representatives of colleges land universities, school district 
administration, and professional staff of schcJol districts. So 
defined, a competence based system permits the widest variety of 
program design since it does not prescribe any set of course or 
' learning activities . A schedule established by the State Education 
Department's Division of Teacher Education and Certification called 
for , all collegiB and universities presently p:|reparing elCTientary and 
special education personnel' to sramit COTipetence -based training 
.program proposal plans by FleBruary f, ia75 to be reviewed by program 
registration. , ' . 



The Regents mandates facilitated the CBTE Project Policy 
Board decision to widen the horizons of the CfeTB Project. Recognizing 
the many difficulties faced by the consortia of over forty degree; 
granting, institutions in this new style of certification; tiie CBT^s 
Project, in collaboration ydth Syracuse University, hosted a conference 
in May, 1974 in Glenmont, New York to mtroduce some ccpnponents of the 
competency based process to members of xhe special "education coinnunity. 
The Proceedings of the fjlrst Glenmont Conference have been published 
in Design for Competence Based Edacatibn in Spfecial Education . 
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As consortia continued to wrestle with the collaborative 
process of program design^ a^hief area o£ focus clearly emerged, 
the coiT5)lex but essential congeries of taslcs involved in assessment. 
The imiltifaceted concept xr£ assessment can anibrace tihe ixtentif it^a^^ 
of competencies, the measur^ent of success in competency acquisition, 
the evaluation of train ing strategi es and programs 4n f acS.litating 
that process, and research to validate competencies. The CBH" 
project has been indeed fortunate in having Dr. Patricia M. Kay as. 
principal investigator of one project component, the City University 
of New York Competency Based Teacher Education Project in Special 
Education which is itself part of the university wide CBTE research 
and develop^pent project. Dr. Kay's unique awareness of the complexities 
of CBTE training, her knowledge in the arpa of asressment, her : 
previous experience with the special education community in New York 
State, and the capabilities of her staff led the CBTE Project Policy 
Board to invite Dr. Kay to arrange a second conference at Glenipont, 
New York on October 29, 30, and 31 as a cooperative venture with 
the State Education Department, Division for Handicapped Children.-. 
-This Dr. Kay graciftisly consented to do. The coiaference and this 
document resulted from a collaborative effort exemplary of the finest 
professional team effort. 

. If the conference was a successful first step down the 
long road to growth in a truly professional grasp of valid assessment 
procedures, a large measure of credit belongs to the sensitive and 
generous -members of the consortia. Typically sensitive to the needs 
of children with handicapping conditions, these members of the special 
education community have given generously of their time, talents and 
professional concerns for improving teacher training. The tooling up 
process towards a competency mode in a collaborative way has been, 
and may be expected to continue to be, a highly complex, time consuming, 
costly, and energy draining process.- It is a tribute to the consortia ^ 
members that this tremendously impressive group of hard working people ' 
brought to the conference intelligence, energy and commitment far 
beyond a desire to meet a mandate, liowever motivating the latter 
consideration. Whatever degree of 'success CBTE may eventually meet 
or fail to achieve, the* CBTE project in special education has found 
a full measure of personal and professional satisfaction in facilitating 
the interactive efforts 'of the consortia members. Both conferences 
in the Walden-like Glenmont atmosphere fostered the personal and 
professional growth which thrives only in the honest, authentic, creative 
and self-renewing dialogue of capable and coimitted people. Competence 
based education is meant, in its best forms, to foster self -development, 
self -renewal, on an individual and program level. The planijers hoped 
that the Glenmont CBTE Conference might be on target with what John Gardner 
wrote of self- renewal! 

K "But the development of one^s talent 
Is only part, perhaps the easiest part, 
of self -development ^ - 
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It is a li£e-l|(^ng process 
that brings ud to the >yecognition 
IsKat the ever/ renewing soeiety 
will be a ivele society/ 
It will un deys^aiid'^ ^t th e-only^ 



stability poksilDle^^^t 
is sta bili t y i r ^-faet-ienv- 



It will foster a climate 
.in which the seedlings o£ new ideas 
can survive and the deadwood 
o£ obsolete ideas can be hacked out. 
Above all it will recognizc|| 
that its- capacity for renewal 
depends on the individuals 
^0 make it up. 

It will foster innovative, versatile, 
ai>d self • renewing men and women 
aa^id give them room to breathe," 

This dpcum^nt recounting some of the proceedings at 
Glenmont II is-of fared in the hope that, like the conference^ 
it may contribute toward the self -development and self -renewal 
which should be hallmarks of any COTipetence based education 
program conceived $nd ntirtured by purposeful people who allow 
themselves the time and room to breathe. 



Joseph T. Gilinore 
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INTRODUCTK^ 



Patricia M. K^y 
Baruchi ColXejje of The 
City University of New Yovk < 



Although teache r educa t ors across the country ar& 



currently struggling with the vast array of complexities 
inherent in competency based teacher education, it is likely 
that nowhere are those issues receiving inore attention than 
in New York State, particularly at those institutions pro-^ 
viding professional preparation for teachers of handicapped 
children. Without doubt, the most difficult of the complex 
CBTE issues. and yet the c^ne that is most central/ to CBTE is 
assessment. CBTE assessment is the search for evidence that 
teaching candidates possess the knowledges, skills .find attitudes 
necessary, for entry into the profession. That statement "appears 
innocent enough - but has implications for initial selection of 
competencies, evaluation of instruction and programming, and 
the validation of competencies as well as for the measureinent 
of candidates' attainment of competencies. 

Assuming that there are no simple answers to questions 
about assessment and that solutions to the problems raised will 
require the attention^ all members of the education community 
for some time to come,nBi^purpose of the "Gleiimont 11" conference 
was to provide a conte^ within which the special education 
consortia from New York Stajte cpuld begin to develop both long 
and short range plans f^r addressing the assessment dimensions of 
CBTE. 

There were four kisifds ^6£ activities planned for th^ con-, 
ference that were designed to assist the thirty- one attending 
consortia in promulgating their o^a assessment plans. There were 
large group meetings designedcyprimkri;iy for input ♦ Robert Soar, 
Melvyn Semmel and Frederick McUonalji spoKe at those sessions 
addressing technical issues of meth^ology, research models, observji- 
tion techniques and results of past research which might form a base 
for conpetency identification, measurement, and ftirther research. ' - 
Those authors' three selections in this Proceedings help' to provide 
the technical assistance dimension so necessary to reliable and 
valid assessment development. Also in large session, Vincent Gazzetta 
spoke on "CBTE Assessment in New York State" outlining the state 
education department's expectations for assessment plans required 
in teacher education program proposals. .v 

The second kind of large group activity at the conference ^ 
is docunented here as the "Contributions and Concerns" section. 
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In these sessions, James G. Ward spoke to the potential contri- ^ 
butions that public school teachers can make to the assessment 
processes as well as the concerns they share about it. Gi'l^rt 
Duken and Michael Solimando addressed the potential contribu- . . 

tions an d-eo nc e ms of -pubHc-schoojb-admini^ 

education students Karen Cochran, Nathan Glasper, Karea Kowalchuk, 
-and-Margar^41a5a^eH formed 

of view about CBTE assessment. The sunomaries o£ all the/'con- 
tfibutions and concerns" presenters help to sort out roles and ' 
responsibilities and, provide many stimulating thoughts that will; 
undoubtedly, become o£ the focus of discussion in numerous con- 
sortia meetings • ; 

Th^HSilrd kind of activity designed to assist consortia,, 
the *development of assessment plans was the small group meeting 
ere members *of consortia could actually sort out the ijssues and 
gin the ^planning processes. Facilitators were assigned to 
ch group and a consortium worksheet was providewd to be uspd 
ither as a guide, a focus for discussion, or a simulation of con- 
^ortjLal assessment plan development. The worksheet used at the 
conference is also included in these proceedings. 

i The final Jkind of activity that yras planned into the 

conference was"^ free time - actually, a kind of non-activity. 
Diiring that unassigned time, participants could used the resource 
center that was available, talk with the facilitators aod con- 
si^tants informally; continue consortium meetings or simply stroll 
und the beautiful grounds and digest the multitude ot ideas 
t were presented as conference menu. 

. How well the conference itself fulfilled it's goals is 
difficult to assess. The real test probably came after the parti- 
cipants left Glenifont and may be somewhat apparent in the assess- 
ment plans- that each consortium submitted to the Division of 
Teacher Education and Certification in January and February, 1975. 
An atteonpt at objective conference evaluation was ^undertaken, 
however, and the results are presented here by Alice Kbrrtblith 
and Cecile Segal. Their evaluation was based on conference objectives 
that were made public at the outset. * 

The objectives of this document are highly similar to 
those of the conference. These proceedings are offered in the 
hope that they will assist readers in becoming more aware of the 
technical, practical, and political dimensions of developing a . 
CBTE assessment strategy and in developing heightened awareness of both 
specific techniques and sources of tecnnical assistance that might 



be employed in moving "toward CBTE assessment^ systans. Additionally; 
readers can find assistance in understanding l^ewYork State's 
expectations for assessment plans as well as the pofdntial 
roles that may be filled by public school teachers, administra- 
tors, and teacher education students. / " 



^ - 
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PROGRAM EVALUATION AND VALIDATION IN PBTE 

' ' - Robert S. Soar ^ ' ^ ^ 
IMiversity o£- Florida 

Introduction 

■ ''^ * • • ' . " 

. One of the critical need^ of the PBTE movement is the 
heed for program evaluation and validation a need which, seems 
uncertain to be met. As Rosner and*Kay (1974) comment, : 

"If the educational community, in general and > 
teacher educators in particular do not fully 
understand and buy into the heavy analytic and 
research deman4^vo£ GBTE, then CBTE vail not be 
realized, Tlie multiplicity of CBTE-related 
definitions will 'expand and successes in the 
name of CBTE will be proclaimed when specific 
or peculiar aspects of the CBTE definitions are 
realized (i-e,, modularization, systematization, 
personalization, individualization, field 
' orientation, etc.). In -that case, the long- 
range reform effort will go down in history 
as another bandwagon in the parade of edtica- 
tional faddism," 
(P- 293) ' - 

Among the concerns that we hold for the PBTE movement are 
that heavy expenditures of money, time and effort may be going 
into teaching student teachers skills which \^dll make no diff- 
erence to the pupils they ultimately teach; that only in un-' 
usual cases, if at all, is the research beiag done which will 
test whether this is true. These concerns also stem from the 
fact that tiae research base we have for teacher education is 
exceeding^ thi^^d that we do not know the, limits of its' 
validity (i*e«, for what teacher behavior, for what objective, 
for what group of pupils). Further, such questions are not 
even asked very often. 

This papef* theii, will present selected fadings from 
our research ovef thp past 14 years in the hope that they may 
add to. the knowledge of the teacliier behaviors which are related 
to. particular kinds of pupil gain; ljut also with the hope that 
they will point to scme^'of the problems of designing evaluation 
and validation plans, J|id collecting and analyzing the data. 

.^j-^- . ■ ■ 



Some Findings with ImpI for 
PrQgram ValidatiQn 



Since the terms evaluatipn and validation will be used 
' invsoifewhat different senses here than is sometimes true, it 

seenis useful to clarify the distinction. Evaluation, as used 
*-here, will refer to measuring the behavior of student teachers, 
at various points in the training program. |0r as they teach in. 
the field, to ^swer questions such as''Tyhe\ther the graduates 
. of this prograan teach differently from graduates of some other 
progranis, or whether the specific competencies toward vdiich a 
program is directed actually are realized, in the teaching be- 
havior of its graduates. Validation, on the other hand, will j 
be used to refer to the question of whether a specific teacher 
behavior has the effect it is believed to have on pupils 
for exaii5)le, dd teachers who emit greater amounts of positive 
affect in fact produce greater subject-matter growth in the 
pupils they- teach J or do larger amounts of teacher -pupil 
interaction at the higher cognitive levels (Bloom's taxonomy) 
result in increased, pupil ability to process information or 
to deal with abstractions? ITiat is, are- the beliefs we hold 
about the nature of effective teaching in fact true; Or better, 
within what limits , are they true? The primary focus here will 
be on validation of teacher behaviors, but since teacher behav- 
iors are the outcome measures for program evaluation, sopie of 
the. findings may also have relevance for that probleim. The 
major emphasis will be on firKiings which have appeared in two 
or more studies . - 

The Inverted ^ ' " 

One of the major ways in which classrooms differ from 
each other is in the extent to which the activities in the 
Classroom emanate from the teacher rather than the pupils. At . 
one e?ctreme the teacher sets^ the problem, directs the activities 
in which pupils are engaged, monitors and reinforces the work of 

^pupils, and evaluates the results of their efforts. Pupils 
have little choice about what they do, how they do it, or the' 
basis on which they will be. evaluated. They have little or no 

"wiggle room*'. At the other end of the scale, pupils have a 
high degree of freedom to choose the activity on which they 
will work, with TVhom, how long, and the decision as to whether 
•it was useful may be theirs, if the question occurs ^t all. 
The teacher^ is. available as a resource, may set outer limits to 
the behavior? which are permitted, but even thesie are likely to 
be broad. Of course these are extremes of a dimension along 
which classtoom scale, with most classrooms somevfliere i^i be- 
tween. " 
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In four sets of data the findings have emerged -that when 
classroowis were raik ordered fron those in whicn pupi^^s have- 
least freedbrn to those in which pupils have most freedon, pupil 
subject-matter gain was Ibwer in classes where pupili had little 
freedom and increased as the amount of pupil freedoin increased. 
But this was true only up to a point, and beyond that point as 
piqpil freedom increased > gain no longer increased but began to 
decrease • This is, there was an optimum point, a balance, 
between teacher control and pupil freedom at which greatest . 
pupil subject-matter growth occuredi (Soar, 1966 § 1968; Soar 
and Soar, 1972; Soar and Soar, 1973); Similar resultshave v ^ 
been found by others (Solomon, Bezdek, and Rosertberg> 1963; 
Coats, 1966; Brbphy § Evertson,. 1974), The same relationship 
has been found for ptpil growth in self-concept in two data 
sets (Soar Soar, 1973). ^ 

^ Although this seems such an' obvious finding as hardly 
to warrant comment, and although it seems clear that good 
teachers have understood this for years, researchers have only 
examined this possibility riecently. And it appears to have 
further ijnplications. Probably the two classroom innovations 
. v^ich are currently being espoused with most vigor are contin-' • 
gency managonent teaching (or precision teaching, behavior 
analysis, or other terms) on the one hand, and open classrooms 
<m the other. It seems clear that these two styles pfvclass- o 
room .management fall toward opposite extremes of the balance ' ^ 
between teacher control aj;id pupil freedom. ] ^ 

The Differentiated 'TJ" *^ / . v 

One set of data, arid to differing degrees two others ^ 
suggest that the balance between teacher control and pdpil^ 
freedom which is associated with greater pupil growth shifts 
systematically with the complexily or abstractness of the' 
learning objective. The general principle appears to be that 
more concrete the l:ask (memorizing the multiplication table, 
today's list of IS^^elling words, or dates in history) the 
greater the degree of teacher control which is optimal; 
but when pupils are involved in complex prdblemrsolving;- 
inferring, abstracting or generalizing, greater pupil fi-eedpm 
is appropriate. (Soar, 1968; Soar ^ Soar, 19Z2; Soar § Soar, 
1973.) Although this is the general principle, for greatest 
pupil growth there still appear to be limits to both the 'V 
teacher control and the pupil freedom which is optimal. 
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Teadher Control in Three Areas ; 

TherQ is limited evidence suggesting that it may be 
functional to distinguish conceptually the control the teacher 
exercises in, three areas: 1) control, of the behavior of 
pupils, 2) control of choice of subject matter, 'and $) 
control of the thinking processes vkich the pupils use. 

*In the classroom in which pupil behavior is closely 
controlled, for, example, pupils might only leave their seats 
to sharpen a'^pencil, to get materials, or to put trash in the ^ 
trashbasket, and in each case would return immediately to 
their seats and resume work- In 'a freer classroom,' pupils 
may move freely from group to group, socialize as they work, 
with limited amount of "horse-play" perhaps even acceptable. 

With respect to choice of subject-matter in the closely 
controlled classroom the pupil works \dth material which has 
previously been selected and- organized for him and his ta^k 
is to learn it. It may be complex, as in arithmetic, or the 
^, -interrelationships' between environmental conditions and the 

nature of primitive cultures^ but he has no choice in what he is 
to learn. In a classroom where choice of siibject -matter is' 
freer, the pupil may collect and organize his own subject 
matter, choose, within liiriits, aspects of interest, and ask 
questions and suggest ideas in classroom interaction. Under 
tighter control of subject-matter, questions are only likely 
to be procedural (details of the assignment, for ex^mpl^) , 
. whereas under less control they are likely to be substantive, 

in the control o£ thinljing, at one extreme pupils 
may be restricted to memorizing facts and giving thm back on 
demand. At the other, they may be encouraged to infer, ^ 
abstract, generalize, hypothesize, to isolve complex prob^CTls^ 
or engage in divergent thinking.^ for example,, pupils whb had 
studied the relation between environment and culture might be 
asked "What would happen to the Navajo culture if thirty 
inches of rain feel each year?" (Taba, 1964>/ or they might 
not go beyond remembering the characteristlcs*«)f a particular 
culture. ' . 

^ The evidence from several studies indicates that most 
teachers do not distinguish among these thfeet area^. In sever- 
al stjjdies, control of behavior vs, corttrol of subject-matter 
and thirling have related in the high seventies to the high 
eighties. If behavior is tightly controlle^^ the thought 
processes and the choice of subject-matter are likely to be 
controlled as well; or if one is*free> all are likely to be 
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free. But limited amqunts of data suggest that the effects 
of freedom* of ^behavior may be rather different from those of 
freedom of thought processed. As an e^^le (Soar, 1966), a 
, measure of "freeing" but orderly teacher-pupil verbal inter- 
action (indirectness as the Flanders system .represents^ it) 
vms positively related ta gain in creativity; whereas a mea- 
/ sure which represented freedom of physical movement in the 
classroom was negatively related to the same outcane measure; 
In ^another study (Soar 5 So^, 1972) teacher control of 
thought processes was related to complex achievement grov/th in 
an inverted 'TJ", whereas a measure of teacher Control of be- 
havior shovred an approximately linear relationship. The 
dis tine tlt)jl between these areas of tea^cher control may be 
worth pursuing in program evaluatiqn and^validation% 

Different Responses to the Same Classroom Experience by Pupils 
^ ~ with Different Characteristics' '' 

Perceptive teachers have been ahead of educational re- 
searchers in the realization that different pupils respond 
differently to the same classroom experience, or that differ- 
ent kinds of experiences are needed by different pupils for 
them to grow. Therp is support. for this concept in the data, 
although it se^s clear that this is an area which needs much 
" more exploration than it has had. (Parenthetically, this is 
a concept, which has re9ently become prominent under the names 
•'trait -treatment interaction'' or "aptitude-treatment" 
interaction". Some examples of this from,our data fallow. 
In classrooms m vdiich teachers were relatively indirect, 
as defined by the Flanders system (that is, they accept, 
clarify, use pupil ideas, and ask questions, rather than lec- 
ture, ^ give directions, criticize or' justify authority), low- 
anxious pupils gained more" in creativity than high-anxious 
pupils did; but in classrooms which were more direct, high- 
4 anxious pupils gained more on the same measure than low-anxious 

pupils did. (Soar, 1968.) Apparently low-anxibus pupils 
owere better able to us^freedom in thinking than were highr 
anxious pupils. 

Another exan^le differential response was the finding 
that disadvantaged (low socio-economic status) pupils responded 
differently from advantaged students to the emotional climate 
in the classroom. They appeared to grow more in the presence 
of positive affect and less ia the presence of negative affect, 
compared with advantaged pupils. Both findings were counter to 
our expeptations^ but perhaps they mean that the disadvantaged 
pupil |g)nore dependent on the nature of the classroom for his 

- V,' 
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•educational progress than. the advantaged pupil (Soar § Soar, 
1973). 

Another characteristic of the child, his degree bf 
motivation to^rd school vrork, appeared tamake a- differ- 
ence in -the nature of organization of the classroorn which was 
associated with mosjt subject -matter gain.' the dimension of 
classroom organization was one in which .pupi,ls were assigned 
seatwork, but when they finished "Were free tp choose other 
activities -- a measure which apparently reflects a mixture of 
structure and control on the one hand, aiid of pupil freedom* 
on the other. Where this patten of classroom organization 
was moreJErequejit, pupils who wbre initially highly motivated ,^ 
shqwjgd more achievement gain during the school year than pupils" 
who were initially low in motivation (Soar § Soar, 1973). 
The reverse also appeared to be true -- low motivated pupils 
gained more where this style of classroqm organization was less 
frequent (presumably because another style was used irj its 
place which was more functional for them). 

'a - , tt ■ ~ 

The Cognitive Level of Interaction Can be -Too High . 

The implication that too much of the ^interaction in the 
classroom can be at too high a cognitive level ^comes frqm 
three sets of data two at the first grade, and one at the 
fffth. * Several dimensions of classroom interaction represented 
the frequency with which relatively abst4»act interaction took 
place between teacher and pupils, following Bloom's Taxonomy 
of the 'Cognitive Domain; ^nd a Deweyian. approach to teaching. 
These measures 'tended to be negatively' associated with gain^in 
both pupil achievement and self -concept. In some cases, the 
negative relatipnships held for the total pupil group, but in 
other cases the negative relationship appeared tp be true for 
disadvantaged pupils but not for advantaged pupils. 

We suspect this may be a finding with particular impli- 
cations for pupils. with learning difficulties, whether the ^ 
learning' difficulties come from a culturally different pre- 
school background or for other reasons. 

Implications for Program Validation . 

Findings such as thes,e conqem us abqjit thfe appro- 
priateness of some of the general prescriptions Which ^e made 
for teacher behavior. Teachers in general are often urged 
to give pupils more fteedom, and are encouraged to assume that 
one of the hallmarks of good teaching is^that the teacher does 
little directing and controlling of pupil behavior. Another 



prescription sometimes is that tiie teacher ought ^to IkqDjiasize 
higher level thinking activities. Both prescr^ions appear to 
be mdq without qualifications about how^^n^ 

iors ar^ functional or jvith wh4t RUj/il "^^oijp they may be - • 
functionary MVHkt is ndt Stated, bi/t is^mpUed,. is that more | 
of these behavioVs is better, witifout IMit)^ better fo^r 
-^aljL pupils and for all learni^^/o^ 

IVliat particularly concerns Vs is that the very teachers 
^ who are more likely to move^ in the direction which is advocated 
are the teachers' who may already be" giving pupils as much freedom . 
as' is functional, (or more), or engaging them in as much higher 
level cognitive -interaqtion as is functional for most pupils. 
In a sense "good" teaclijers are more, ready to become "better" 
teachers , and we suspedt the same thing is true about schools 
and school districts. Ilie possibil:^ty seems very real that, 
•teacher preparation prdgrams aijd in-service programs may' expend 
considei^able effort tovlard changing the behavior of some 
teachers in directions Iwhich may result in less pupil growth 
rather than more . - . ^ - 

Scmie MQthdJfiologicall Issues in Research and Assessment 

in PBTE • 

We Ijave cqrinente|d earlier ab<iut the importance of beginning 
to assure ourselves tiiait the teaching behaviors which tea.ch 
are really functional for the growth of pupils, Contr^ictions " 
in the kinds of teaching vihich are advocated are easy to find, 
anH the principles whiqh we thought were effective fairly of ^en 
do not stand empirical tests. V.Tiile it is clear that the oi^n- 
ization of training programs cannot wait for researchers to^ ^ 
answer questions of this sort, it seems nonetheless criticaJJ^ 
important that training programs begin to test , in the context 
of the programs, the validity of the principles they teach/ There 
is a limited amount of evidence in the research literature which 
can help with this process, both by providing SOTie concepts 
which appear to have been replicated, and also by suggesting new 
concepts for investigation which might noti other\dse be considered. 
It seems important to cycle new knowledge of teaching effects 
back into preparation programs tor revalidation --a kind of 
"bootstrap" process. 

Another suggestion has to do with the analysis of the data. 
By far the greatest number of past Researches in education have 
been based on correlations and t-tests. It seems clear that 
the relationships between classroom interaction and pupil 
growth are too complex to be represented adequately by such 
simple analytic ptocedures. Rather, it must be possible to 
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loQk at the spiultaneoiis effect of multiple influences. It 
must Dfe possible, fpr example, to examine several character- 
istics of the cliilS as he enters the classroom, including the 
^' home and broaden coimrunity, along \dth numbers of aspects of 
r^his classrocxn ej^erience, as these affect the changes the pupil 
makes. It is aj^o necessary to look at non-linear relations, 
since the evidence is compelling that these ar^ common in this 
area of research, s While these requirements are overwhelming if 
conceived of as cejls in a multi-factor analysis of variance, 
recent advances in* the use of multiple regression (Cohen, 
1968; Kelly, Be^s and McNeil, 1969; Walberg, 1971; Kerlinger 
8 P©dhazur, 1973)« make it relatively straightforward, using 
• programs that are available at most computing centers, tf -the 
skills and the degrees of freedom to use multivariate proced- 
ures are availjable, 'fetill greatei* increases in power and in 
understanding becdme possible. The cost of these complex 
analyses in terms of effort aryi money is minor, but die gain 
in knoi^ledge is tremendous. 

But the point which seems most critical is that major 
expenditures are being invested in program development and 
implementation with little or none invested in program vali- 
dation. If this continues to be true, as Rosner and Kay 
commented, the effort is likfely to come to nothing. We will 
not know, finally, whether the teaching behaviors which have 
been taught make any difference to pupils, and consequently, ^ 
we will not know whether the program effort lias accomplished 
anything. Too many past innovations in^ucation iiave fallen 
by the wayside with no evidence of their success or failure. 
We hope for better than that for the PETE movement. 



■ 20 



ERIC 



REFERENCES 



Brophy, J. E. § Evertsorj, C. M. . The Texas teacher effect ivenss project ; 
' Presentation of non-linear relationships arid simrnary discussion , 
Report No. 74-6. Austin: Research and Development Center, 
University of Texas, 1974. 

Coats, W. D. ' Investigation and simulatiorf d!^ the relationships among 
selected classroofn variables. Unpublished doctoral dissertation. 
University of Michigan, 1966. Cited by Flanders, N. A. Anajyting 
teacher behavior . Reading, Massachusetts: Addisen- Wesley, 197n. 

Cohen, J. Multiple regression as a genersll data-analytic system. 
Psychological' Bulletin , 1968,- 7f>, 426-443. ^ 

Kelly, F, J., Beggs, D. L., § McNeil, K.^A. Research design in the 
behavioral sciences: f4ultiple regression approach . Carbondale: 
Illinois University Press, 1969. 

Keflinger, F. N. 5 Pedhazur, E. J. Nfultiple regression in bgbgvioral 
research . New York: Holt, Rinehart ^ Winston, Inc.^ 1973.^ ^ 

Rosner., § Kay, P. M. Will, the promise of^C/PBTE be fulfilled? 
Phi Delta Kappan , 1974, 55, 290-295. 

Soar, R. S. integrative approach to classroom learning . NIMH project 
numbers f-Rll m 01096 to the University of South Carolina, and 
7-Rll MH 02045 to Temple 4Jniversity, Philadelphia, Pennsylvania, 
1966. ERIC document No. ED 0^ 749.^ 

Soar, R. S. Optimum teacher-pupil; interaction for pupil growth. Fduc. 
Leadership Res. Supplement , . 1968. 1, 275-280^ 

Soar, R. S. § Soar, R. M. An empirical analysis of selected Follow 

Through programs: An example of a process approach to evaluation. In 
.Gordon, I. J. (Ed.) Early Childhood Education . Chicago: Mat. vSoc- 
for the Study of Ed., 1972, 229-259. ^ V 

Soar, R. S. 5 Soar, R. M. Classroom behavior, pupil cliaracteristics , 
and pupil growth for the school year and for ths summer. lAiiver- 
sity of Florid?, grant numbers 5 RO 5 MH 15891, and 5 R05 ffl 15626^ 
National Insti\tute for Mental Health, Dept. HE1V, Washington , D. C, 
1973. ' 

Solonon, D. , Bezdek, W. E., ft Rosenberg, L. Teaching styles and learnyrig , 
Chicago: The Center for Study of Liberal Education of Adults, 1963. 

■ ' 21 . • 

■ . ' -iz- 



Taba, H. , Levine, S., 5 Elzey, F. ^. Thinking in elementary school 
children. Coop. Res, Pro j . No. 1574, OE, U. S. Dept. of HEW. . 
San Francisco: San Francisco State College, 1964. 

Walberg, H. J. Generalized regression models in educational research. 
Am. Educ. Res. Jour., 1971, 8, 71-91. 



1 ' ' . 22 

-13- 



r 

22 

> 



^ CBTE ASSESSMENT IN NEW YORK STATE 

Vincent Gazzetta 
Division of Teacher Edncatjon and Certification 
New York State Education Department 

I just happened to be on my way from Goshen to Albany, 
heard you were here and dropped in. IVhen I arrived and was 
asked to ^end an nourwith you 'I was not only flattered, but 
happy to do so. 

The reason for my happiness is that I overheard what I 
took to be a compliment after I finished the last time I -was here. 
I overheard sOTieone say after my remarks ikst Spring - 'That 
speech reminded me of what George Bernard Shaw said about IVag- 
nerian music - 'It's not as bad as it sounds."' 

Tliat remark overshadowed a caunent representing the other 
side of the coin, vAien someone said, "Every speech you've given 
is , better tlian the next.'' 

The Saturday before last I had the opportunity to spend 
a couple of hours with the New York State Student Education 
Association. I guess there were sme 90 college students in 
attendance and whenji asked how many were -knoi^ingly involved in 
CBTE programs, I'd say that about ten of them indicated that 
they were. ^hJ \vas surprised and glad.' Surprised because these 
young folk were seniors arid I didn't expect that many seniors 
to be involved. ' I 

Unfortunately, I spent more time talking at them than 
with them, but the time I spent doing some probing of their im- 
pressions o£ CBTE was well spent. I found that they did know 
what the elements of the program'^ were sSAd that they did know 
the time allocations for program parts. I found that they gen- 
erally felt the programs were more difficult, but also that they 
were pleased with their programs . 

» The one problem that came out, and I admit I dug for it, 
was the cpiestion of assessment. That should not surprise you, 
and it didn't surprise me, for assessment is one of our biggest 
problems and one on which time and effort will continue to have 
to be expended. 

There are basically three kinds of assessment and I want 
to talk about them ^ little bit' this afternoon. First, student 
assessment - by the way, student is our term for a person in 
training. A pupil, in our terminology, is a K-12 enrollee. By 
student assessment, I mean the assessment of the student's cap- 
aiw-lities. Second, program assessment - which searches the 
question of whether tlie program is doing what it set out to do. 
The thirds is another kind of program assessment aod relates to 
the State's assessment of program quality. 
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In looking at these three I want to use the third , as the 
operational base. The^ Department's responsibility is to'do^^l 
that it can to assure the public o£ the quality o£ the program. 
And*, I have a sneaky suspicion that you are interested in what 
"they" will do in regi^gtering preparatory programs and issuing 
certificates. Who was it that said "We have met the enemy and 
they is us"? I guess maybe that fits in this instance. 

^ r^The Department will be assessing your program tluree times. 
That's the registration 'procedur^-preliminiary registration upon 
submission of a proposal, condition^^ registration is the second 
step,* and the third step is contiraing registration. Now, that 
third step, continuing registration, will be repeated on a per* 
iodic basis qf roughly every five years. 

The first assessment will be what your program proposal 
goes through when received in Albany. It will be assessed on 
completeness and on the content. TTie completeness assessment 
relates to whether the items recpiested in the format have, in 
fact, been submitted. ^ If items ^re missing, we will ask for 
them before taking that proposal any further. 

The content assessment will be one vdiich prim^ily assesses 
existence. For this is a new world for all of us ^d our goal 
during these early years is to establish the conditions from, 
which needed improvements can be identified and subsequently 
acted on. ^ j ^ * 

I won't take your time to go over dll the specifics on 
which we will judge the proposal, for those specifics are noted 
on pages 8 and 9 of the program format material* which you 
have in your folder. I do, however, want to talk about a few of 
the items in that section of the format. 

First, I want to reemphasize the importance of the re- 
quired analysis and statement of the position for which persons 
are being prepared by the proposed .program. While we are riot 
going to assess the quality, validity, or feasibility of tliat 
conceptualization, we shall be seeking its iiexistence as the 
underlying principle on which the program is built. There 
should be clear evidence of congruence between the conceptual- 
ization, the skills, knowledge, and attitudes, and the assess- 
ment of the possession of » those desired capabilities. 

Peter Aerasiaja, in writing about evaluation issues, quite - 
clearly presented the rationale for the need of a conceptualiza- 
tion when he wrote : . 




*Format for Submisspn of Teacher Education Program Proposals issued 
by the Division of Teacher Education and Certification, New York State 
Education Dep.artment. o^i 
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Th^ heart o£ the evaluation process is 

valuajig. Date gathering, be it 'Tiard/' objec- 
tive ^^ta, or "soft," impressionistic data, 
is not evaluation. Evaluation takes place 
v^en datavare compared to -some standard or 
norm and av decision. • .is made. 



In another party o£ his paper he says it more clearly. 

While some observers argue that the most power- 
ful individuals in a performance -based approach- 
are those who ultimately certify performance or ^ ' 
competency, I would urgue that the most powerful 
individuals are those who frame the competencies 
to be attained. These are the individuals who 
explicitly define what is a good teacher. The ^ 
decisions of these individuals color the selec'tion 
of learning experiences^ as well as the evaluative 
techniques and criteria*. In the performance -based 
approach, which proceeds from identification of 
ends to selection of means %o obtain these ends, 
if is the ends which are paramount. The rationale 
for a program, its learning experiences, standards, 
and certification practices rest Jjpon the perfor- 
mances defined as needed by th^g^od teacher. 

Thus, to us^the- establishm^t of the^cwicepttialization 
is the creation by the collaborative effort of the value 
standard. This explains our cofnment on page 6 which says: i 
"The conceptual izatioiwrTshould be... the key faptor in the 
program degree " 



A second point I'd like to make reference to is the 
assessment of students. -^Here^^gain we are at the cutting 
edge. Sophisticated, valid, and reliable instruments foi; 
the variety of assessments necessary are not available at 
this time. And I'll hasten to add that if we hold off doing 
anything until these assessments are available, we'll be at 
the same place in the year 2049 as we^ are now. 

The development of"^e necessary sophisticated 
assessment capabilities can only be' done in the context of 
real life situations. And, I believe, they can only be .done 
within the context of the value orientation established by 
each program. ^' 

What, then, is necessaiy at the time the proposal is 
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rcvie^ved by ^he State?. First of all, is the need for there to^he 
congrifence between the expected capabilities and the assess- / * 
ments. Second is 'the explicitness of the assessments, . I^t is. ^i 
really not enough to say, as one institutional progr^ proposal- 
noted, "A satisfactory performance will be required." We andf . 
you both need to know what "satisfactory" is and inider.whatv $»| ^ 
conditions a person should demonstrate ^satisf^tq^iness^' » "J?; 

/ » • ^ , ' 

Let me try ray hand at a very rough and si^le dJcanjRje. ^ 
Using the competency - \ * 

The student will demonstrate at least three ways ' 

of helping retarded djildren learn- and identify v 

the following parts 'of the body: eyes, ears, . - - ^ 

nose, mouth, luinds, and feQt/ -Each means used 

shall also aid an developing largd muscle cpprdi- ' . ' ^ 

nation. > ' , 

Now, a statement of assessment might be: ^ ^ ' 

The. student, in a pegr- teaching setting shall • 
demonstrate attainm'ent'^of the competency by 

using at least two of the follovdngi three — '7 A 

means: a physically active gas^, rhythnic ^ 

song and appropriate body moveiT|eShts, gross V^,, v-'^-^i^' 

drawing skiB-s, . - ^ 

: > In this simple skill example I've tried to be as expy.c^^\ 
as possible and identified the conditions - in this case, Xhe\ 
condition is the "peer-teaching- session" - and the mastery' level, ^ 
in this case, 3 ways to include 2 out of 3 stated ways. ^ 

The' more we move away from skills the more difficult it 
becomes; thus, the more important that the three pertinent 
agencies are involved. We cannot yet expect perfection, 
but we can ask that you identify what your criteria are for ^ 
satisfactory demonstration of the skill, knowledge, and/or 
attitude. When do you know a student has attained the capab- 
ili,ty'you expect? That's the question which needs to\be' 
asked, answered, observed, modified and refined as necessary. 

Other aspects of assessment will be reviewed when your 
proposal is received. We will look at the degree which your 
assessments arc public - how you attempt to keep Students -in- 
formed of wliat they need to do. ^ 

We, will look to see that you have laid plans to deter- 
mine reliability and validity of your assessments and that you 
have identified the means by which you will monitor aurid evaluate 
your program. * . 
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You should be aware of what steps your proposal will go . 
through when it reaches the Department, \ 

Upon receipt of your proposal, it will, be acloiowledged and 
then reviewed for cqnipleteness . The proposal is then assigned 
to a staff mmber of a thorough review. Th^ person who will be 
assigned to do the review will not be the ^e st^f member that 
serves as a liaison person for the particular institution of 
higher education, the review by staff also calls, for sharing 
the proposal and se,eking commentary and jrecommendations from, 
the Division for Hamdicapped Children. This total review 
culminates in a meniorandum commenting on the programs strengths 
and weaknesses ang it recommends a course of action. > 

■ ■ ■■ / ■ ^ ■ 

The proposal is then subjected to scrutiny^ by a Review 
Panel consisting /of the reviewing staff member, the liaison 
staff member, and the Chief of the Bureau ,pf Teacher Education. 
It is, then, the, review panel that acts on the reviewers 
proposed course ^'of action ^which would usually take tpne of 
tm avenues. ]pither a recommendation for legal registration 
action is forwarded to the Associate Commissioner accompanied 
by a letter Qf /commentary or the proposal is sent back for v 
further investigation by the staff. 

When a preliminary registration based on the proposal is 
issued, it wifl. contain a termination date and, in the case of 
programs with, which you are involved, thit date will mojpe than 
liKfely be September, 1979. . 

This Mean's that your program will be expected to have 
achieved the second stage- -conditional registrationr-by that 
time* 

At a mutually agreeable time during the four year period 
the program wiU be visited. The visit shall serve to collect 
some data knd also to ascertain the readiness for^ the program to 
be granted conditional registration. 

In terms of data gathering we will be seeking data about 
items such as: . • * 

the effectiveness of the monitoring system related to 
studenj:-"^idance and rejcord management. 

what changes have. been made in the program and what 
data occasioned the c^ianges to be made. 

• * 
the extent and effectiveness of inter-agency colla- 
boration. 

While these items will have a bearing on further, legal 
registration action, the two major areas of review for condition- 
al registration proposals will be: . ^ 



a) 
b) 
c) 



a) the iinplCTientation and effectiveness of procedures 
to determine reUability and^ validity of t|T^ \ 

\ assessments. ^ ' 

b) the initial implement evaluation and . - 
program modification system. 

Iflien conditional^ registration IS recommended and granted 
by the Associate Ccnwiissioner another terminat ion date, probably 
fetfo yeaifs from conditional registration will be identified. By 
that termination date, continuing registrktion is to 
achieved* Cpntiniiing registration will be awarded for five year 
periods only. 

Continuing registra^tion will be granted when evidence is 
provided that the pro^c^ monitoring and evaiuatipn syst^ is 
capabldl of doing at least, four things: ' ; 

a) perceiving. deficiencies in the ccHnpetencies of 
graduates. , ^ ' ' ^ 

b) effecting timely modifications to rectify deficiencies. 

c) validating competencies and assessments. 

d) increasing the reliability of assessments. 

The item I *ve mentioned relating to what is necessary for"" 
conditional and continuing registration are, I agree, very 
general in nature. I will promise you that by February 1, 1975, 
and I hope earlier, we will have clearer and more explicit 
information about the second and third stage registrations for 
you. ' 

^ Before-!.* stop I want to mention an it^ close to my 
heart ^ that is related to assessment but not exactly in the 
same sense that I, have been talking about. 

There are several populations that certification deals 
with and all'^of them, I j^hif^c', must be included in your 
thinking agd planning.. -They aire: 

1. The person who begins and cairpletes an entire 
registeredprogram. , 

2. The person who transfers into a registered program 
from a two year institution or another 4 year program.- 

3. The person who is now a paraprofessional in the schools. 

4. The person certified in one area but who wishes tp gain 
certification in teaching 'the handicapped. 
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5. The person tvho is not certified, but has experience in 
a non-public school • 

6. The person who has canpleted a preparatory program out- 
, of -state or out -of -country hut who is not eligiblei for 

reciprocity, x;^ . . » 

7. The person who was certified, but whose certificate . 
' has lapsed. ' . ^ 

8. The person who has completed a set of courses at 
various institutions who may not quite meet cur- 

w rent requ;Lrements or who applies after the deadline. 

^ , Up to now preparatory programs have generally dealt with*^ 
persons in categories one and two and the State, on an individ- 
ual basis, has dealt with the other six populations. 

Our announced plan to go out of the individual evalua- 
tion business as of February 1, 1976 for elementary and special 
education which I , believe in very strongly, throws, a respons- 
ibility on those involved in preparation programs. While we 
have some time before the full iiifipact hits us .in this area, it 
is necessary for us to look af ways of providing fair and hon- 
est treatment for those populations who ard not regular members 
of registered programs. . 



In finishing, let me commend three, publications to you. 

1. Thfe I^ilti-State Consortiimi publication entitled 
"Assessment." 

2. The AACTE monography #14 entitled "Performance-Based 
Teachea: Education Desi^ Alternatives: The Concept 
of Unity" which is just out. 

3. The September issue of the CBTE newsletter. TTie, 
lead article, "For Want of -An Assessment System CBTE 
Programs are Lost," is important. 
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APPLICATION OF SYSTEMATIC OBSERVATION TO TEACHER EDUCATION 

Melvyn I..Senimel 
Indiaiia University . 

. The training of personnel to work with handicapped 
pupils in the public schools: has been the target of ^some 
professional attention (Blatt, 1966; C^ickshank, 1967; Deno, 
1973; Lilly, 1971; Shores, Cegelka S Nelson, 197^), but pro- 
bably has not received the scrutiny warranted by the topic ^ 
Training programs in special education have most recently 
been influenced by the general trend toward performance-based 
teacher education (PBTB) . The evolution of new training^ _ 
programs in special education is characterized by an emphasis 
on the specification of training. objectives in behavioral 
terms and the establishment o£ criteria for the assessment 
of their attainment (McKenzie, Egner, Kiiight, Perelman, , 
Schneider, S Garvin, 1970; Meyen § Altman, 1973; Shuster, 
1973; Schwartz, 1971; Schwartz, Oseroff, Drucker a Schwartz, 
1972 Shores, Cegelka S Nelson, 1973). Alternatives to trad-, 
itional training procedures are taking the' form of self-contained 
modules designed to furnish trainees with a defined sei. ot 
objectives for attaining speqific knowledge, attitudes and/or 
skills (Thiagarajan, Semmel- S Semmel , 1974). While such 
modules appear to be particularly promising approaches to 
developing specific, noninteractive teaching competencies 
fe.g., planning for instruction, grouping, assessment ot pupii 
ability levels, etc.) , relatively few programs have developed 
the means for assessing and developing interactive skills in 
practicum environments (i.e. behavior management techniques, 
questioning skills, motivational techniques, etc.). 

Observation instruments have shown considerable promise 
- as basic teacher training vehicles around which a coirprehensive 
program for the development of interactive teaching skills can 
be realized (Semmel S Thiagarajan, 1973). Essentially, observa- 
tion systems are used to record objective information on subse- 
quent interactions with pupils. Finally, methods of analysis _ 
freatiently necessitate reduction of observation data. Consequently, 
man? relevant dimensions of the observed pupil-teacher, interactions 
fe g sequential patterns) are obscurred- frequently resulting 
in'an'oversimplified dekription of the behavioral interchanges 
between trainee and pupils. Until these limitations can be overcome 
iS a cSst-feffective SaJmer, the practical application of systematic 
observation methods to teacher training special^ducation will 
remain unlikely- -and the development of PBTE progrdins with emphasis 
on interactive skill development in practicum settings appears 
more as a promise than a feasible reality in the immediate 



future. 
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The Computer-Assisted Teacher Training System (CATTS) 



the application o£ cpmputer technology offers a 
promising splution to a number of existing limitations in the 
use of systematic observation techniques in teacher education 
programs. Special educators are currently exploiting opportunities 
afforded through recent advances in technology* Application of 
computers in special education is broad in scope, ranging from 
direct computer -ass is ted instruction (CAI) of handicapped pupils 
(Stolurow, 1960), to offering ongoing branched CAI special 
education courses at the University level (Cartwright, Cartwright, 
§ Robine, 1972), to providing state -wide computer managed 
curriculum objectives and materials retrieval systems for prac- 
titioners in the field (Noffsinger § Daike?*, 1972). Undoubtedly, 
as such efforts progress and;/cost factors are. controlled, we 
will be faced with the reaixty of a technology revolution in'^ 
special education within the coming decade. ^-J^ 

The question which arises then is, "Can we utilize 
computer technology for preparing teachers to work effectively 
with handicapped pupils in special and/or regular class settings?" 
To my knowledge, few att^npts have been made to explore and 
evaluate the potential contributions of computers in developing 
observation procedures for iinproving trainee interaction skills. 

Much of the writer ' s * research and development activity 
is being directed toward realizing a cost-effective Computer- 
Assisted Teacher Training System (CATTS) (Semmel, 1968; Semmel, 
Olson § Weiske, 1972; Semmel, 1972). CATTS is conceptualized as 
a closed-loop cybernetic system capable of providing continuous, 
instantaneous, and/or delayed feedback of relevant teacher-^ pupil 
interaction data to a trainee in the classroom in order to modify 
behavior through regulatory teaching moves. These moves are 
determined in accordance with predetermined training objectives: 
The system (Figure 2) is designed to produce a cost-effective 
means of collecting data from systematic observations and real- 
time analysis, storage, and feedback of information relevant 
to pupil-teacher interactions in special education contexts. 
Feedback can be provided through instantaneous visual display 
in the classroom or through hard-copy computer printout immediately 
following an observed teaching perfonnance. The system permits 
rapid analysis and accumulation of stored data 'within and across 
teaching situations. CATTS also provides for computer managed 
techniques for building and adaptirjig observation systems and 
an efficient means for training reliable observer-coders. The 
system is designed for application in both preservice and inservice 
teacher training contexts. It can be used within a teacher 
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education laljoratory on a -college- or university campus, or 
directly in public school classrooms In the comramity. Computers 
access is available through direct on-line interface, Touch- 
Tone telephone interface from remote locations, or through- 
offrline magnetic tape devices called DATAMVTES. Hence, it can 
readily be seen, that GATTS is a promising a^jproach to eliminating 
a major deterrent to the* use of systematic observation procedures 
in special education teacher preparation programs. 



A Preservice Teacher Training Model , " , 

CATTS is used in, conjunction vdth an evolving training 
model in -which teacher education is conceptualized as a problem 
in adult learning . The learner is required to generate teaching 
behaviors appropriate to the educational context in which he \^ • 
practices. Teaching is viewed as a performance sXill which is 
best learned by practice in training settings ^ with aqcur3,te 
and papid feedback of performance being essential to e'fficient 
acgj&sition of goal behaviors. Efficient acquisition of 
t^hing skills is dependent upon (a) the specification of target 
behaviors, (b) reliable and valid feedback of performance informa- 
tion during or inmediately following acquisition trials, and (c) 
access to data from pr^ous training trials. 

The model currently guiding our training research 
efforts focusses on the trainees atquisitigri of "appropriate" 
interactive skills through a process of disgrminating , generating , 
and evaluating interactive behavjots, patterns , environments . . 
The progtess from simulated conditions, to practice in a cQ^rolled 
classroom laboratory, to the naturalistic context of comrajnify , 
schools. Ine discrimination process is defined by the trainees' 
acquisition of the ability to build, adapt, and/or adopt an' 
observation system that operationally defines the domain (s) of 
interest, and by the trainees' ability to reliably apply the 
system in observation-coding training sessions. Once proficiency 
in discriminating the categories^ of the observation system is 
demonstrated, trainees are provided with reliable feedback of their 
teaching using the "lexicon" and "grainnar" of /the system selected. 
Behavioral objectives are derived from an analysis of such 
basei^e feedback and the trainee moves into the generation phase 
of training. Generation refets to the performance of specific 
behaviors, patterns and/or enV-ironments* generated by the trainee 
while teaching in ^ appropriate training context. During the 
generation phase, the acquisition of skills is reinforced through 
the use of feedback of information to the trainee during and/or 
ijimediately following any training .trial. The parameters of , 
feedback vary as a function of the objectives of any training 
trial or sequence.- Following feedback, trainees are encouraged 
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to evaluate their performance on the basis o£ objectives estab- 
lished for the pupils being taught. Hence, the evaluation process 
includes (a) assessment of the degree to which teaching skill 
objectives are being achieve^ arid (b) the degree to which the 
modification in teachjjig behavior is correlated with desirable 
pupil behaviors. Trainees proceed developmentally through the 
, training ^radigm by first focusing on the mutually exclusive 
categories which define the subordinate structure of the observa- 
tion system— then moving to more complex interactive sequence * 
which define teach^ig patterns (e.g., pfobing ; teaching question, 
followed by pupil response^ followed by teacher elaborated 
question, followed V pupil elaborated response). Finally, '"^ 
students attempt to discriminate,! generate,' and evaluate what 
we refer to as pedagogical environments. Such environments 
are conceptualized through the analysis of patterns across a 
given teaching episode.' Pedagogical environmMts are exemplified, 
by the various models for teaching described recently by Joyce S 
Weil (1972). 1 ' / / / ^ - 

The training model idenllif ies three training contexts - 
in which acquisition of skills can occur: simulated, controlled 
(laboratory) , and in situ . There is some evidence that -training- 
in the natural environment may be inefficient and/or' unfeasible 
for particular interactive training objectives. Trainers are . , 
frequently unable to control the occurrence o£ events necessary 
for discrimination pr generation skill development. Further, 
trainees are frequently faced witih administrative and ethical 
issues in proposing to develop all trainee interactive skills 
within the context of the public school. 
- . ■•■ i' . . •■ 

The interacting skill tbmponents of the evolving teacher 
training model operate within th6se behavioral domains deemed • 
most relevant to;; working with hzmdicapped pupils. Hence, the 
domains within which conceptual' hierarchies are most frequently 
considered are: cognitive/acadeiilic, affective/management, social/ 
participation, and linguistic/cOmmunication. Other considerations' 
in operationalizing the model include: the nature of contextual" 
and other input variables and n6ninteractive teaching skills 
(e.g., assessment, planning, gprouping, etc.)., V ' 

CAtrs is the principal. vehicle'. through whith we are 
attempting to dl^nstrat6 the various elements of the teacher 
ttaining jnodel in preservice special' education programs. The 
system provides ti§ with autoirtated techniques for rapid discrimination 
training', rapid cdilection ajid analysis of interactive behaviors 
and patterns, rapid feedback of information to trainees, and 
rapid storage and retf ieval d£ ail training trial data for 
all trainees in the' program. The system has the capability 
of processing data generated from any behavioral domain' and cdn 
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simultaneously acconmodate up to 12 different observation 
systems. The data can be subdivided into N mutually exclusive 
categories. The system further permits data entry from any 
or all of the training contexts described above- -including 
direct on-line entry and feedback from and to remove practicum 
settings in the camiunity. 

Preservice Research and Development Activities vdth CATTS . 

Initial work on the Computer -Assisted Teacher Training 
Systems (CATTS) was reported by the vnriter (Semmel, 1968) and 
his students, Kreider (1969), Weaver (1969), Schmitt (1969), 
and VanEvery (1971). In general, .these studies support, the 
efficacy of Immedf-ate concuirent CATTS feedback when specific 
behavioral goals are central to training. 

Schmitt (1969), for example, used the (SAtTS system 
and a -modified version of the FIA system to train preservice 
teachers to increase their use of broad questions and reduce j 
the frequency of binary questions in teaching the EMR.. The 
results indicated that CATTS trainees spent significantly more 
time' asking broad questsdons than did control group trainees. 
The study also indicated a positive relationship between teachers* 
use of ^broad questions and the- production of multiple -word > 
responds by EMR pupils. * ' 

VanEvery (1971) used CATTS technology to study training 
of speech therapists in, a clinical setting. A remote telephone 
line was used to .communicate between the speech clinic and the 
CATTS computer facility. Observations of therapists in training 
were coded in the clinic and 'transmitted by telephone line to 
the computer, which fed back information in real-time. The 
feedback was presented on an event recorder which traced a pattern 
representing training objectives on a moving belt of paper. 
Trainees who received the immediate CATTS feedback showed a 
significant increase in the use of social reinforcement patterns 
when compared to a control group. VanEvery* s* work demonstrated 
the feasibility of eventually moving CATTS into remote public school, 
classrooms for inservice training opporti^ities. 

Merrill Sitko and I have recently completed a study 
within the context of our special education methods practicum 
for undergraduate trainees (Semm'el 5- Sitko, in preparation) ; Our 
purpose was to demonstrate the effectiveness of CATTS" inmediate 
visual and delayed (printout) feedback in increasing various 
/cognitive and management behaviors of teficher trainees in a 
classroom setting. Trainees received either ii^tantaneous -continuous 
feedback through a TV monitor while teaching and a hard-copy summary 
printout ininediately after a teaching session, or received only 
the hard-copy summary printout. Skill objectives wtre self -selected 
by trainees. A variation of the replicated single -organism 
multiple baseline design was used in the study. 
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The results revealed that" all trainees in both treatment 
ponditions significantly increased their Baseline criterion rate 
o£ performance as a function of CATTS feedback. In fact, the 
relative rate of responding to the selected criterion measure 
between feedback treatment and baseline phases was approximately ' 
4:1. Trainees who received both the CATTS in-class video 
and hard-copy delayed feedback increased their criterion 
rate of responding to a significantly greater degree than^did 
trainees in the delayed printout -only condition. The relative 
rate of responding ^between the two groups on the individually 
chosen criterion measure was approximately two to one (2:1). 



Observer Training Through DITRMA 

"The discrimination skills of trainers are developed 
through the training of reliable observers. A newly developed, 
computer-aided training device called DITRMA aids in the develop- 
ment of these observation skills. DITRMA is based upon a simple 
consensus coding principle wherby individual trainees' responses 
from two or more coding terminals are simultaneously ccympared 
by the computer, and the result of such comparison is instantaneously 
fed back to the trainees. Through expanded application of this 
simple configuration, the DITRMA system can be used to teach 
discrimination of relevant teacher ^pupil behaviors, and to 
maintain that level of reliability^ DITRMA is a second generation 
consensus coding system orig'inally developed by Semnlel, Guess 
and Flanders at the University of Michigan. 

During coder training sessions, observers' code video 
taped examples of the observation system categories on button 
boxes having a configuration identical to Touch-Tone telephones. 
These button boxes are "linked to our PDP-12 computer. The' 
computer acts as an impartial judge as tr-ainees code with a 
small group of up to five peers. If afl coders agree vdth each * " 
other on the coding of an event, the group receives a visual 
and auditory reinforcer from a second video monitor and loud 
speaker (e.g., "riglat-on'', ''Great,** "Good," etc.) the videotape 
continues^ and the computer records the agreement in its memory • 
bank. However, should one or more coders disagree with the 
others, the computer automatically stops the videotape, and a ^ 
display of all identified codes appear, oh the second video 
monitor for all trainees to study. Trainees subsequently discuss 
their differences-^ the computer "refuses" to continue the 
first videotape monitor until the group reaches a consensus of 
agreement on what is the correct discrimination and code. 

DITRMA was first used in connection with the previously 
described Semmel and^ Sitko (in preparation) project. Evaluation 
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o£ the sytem was very promising. Coder training time was reduced 
by approximately 50% when compared to our previous procedures 
using paper and pencil techniques. 'Of perhaps greater importance 
is the serendipitous realization that DITRMA is a potentially 
powerful device for training many teaching skills through 
' what, in 4f feet, is an automatic, self-instructional group 
-€ormat. The system acts as an impartial and automatic discussion 
leader for small groups of trainees who are viewing videotape 
protocols.. We are currently developing a number of prototype 
instructional training programs which use DITRM/V for the 
. training of noninteractive teaching skills, attitudes, and 
knowledges . 

The new DITRMA system is also capable of assisting in 
the development of category observation systems. We have, found 
DITRM/V invaluable for rapidly identifying categories which are 
subject to relatively high rates of observer disagreemCTit^-an 
indication that the categories are imprecisely defined. 

> Application of CATTS to Inservice Teacher Training . ■ ■ 

The application of CATTS to inservice training bf < 
teachers has particular promise for contributing to more 
effective teaching of handicapped pupils. Existing inservice 
training paradigms most frequently require the teacher to 
"come to the training". Courses are offered at local colleges 
and universities, or workshops are offered by visiting consultants ^ 
These usually require meeting at some central location -avay from 
the classroom and school. Inservic^ activities offered through 
school districts usually require after-school attendance by 
teachers ^who are unmotivated and exhausted after a hard day s 
work with pupils. More and moVe, we are finding the issue 
of inservice training requirements -of school systems the subject 
of collective bargaining in the negotiations of teacher's 
contracts. It is well known that inservice training after the 
working day or prior to or after the school year is becoming 
more and more difficult and costly. Further, attempts to 
improve teaching skills through direct supervisory classroom 
visits are frequently fraught with difficulties. Supervisors 
rarely use systematic observation approaches, and the results 
fed back to teachers are often haphazard and unreliable. 
Supervisory feedback is often vieifd by the teacher as a . 
subjective evaluatibn with potentially negative consequences; 
this in tttnv-raay lead to increased defensive behaviors with 
si^aervisors and/or' socially acceptable reactive effects to. 
being observed. 
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CATTS provides a possible solution to many of the ' 
above problems since it is a system of inservice training 
delivered to- the teacher directly iii her own classrooip, during 
the school day, as she works with her own pupils within the 
context of a curriculum of her choosing. The system has the 
capability of delivering nonevaluative, reliable feedback to 
the teacher about her interactions with pupils. Feecfback can ^ 
be provided with a minimum of delay, thus maximizing the 
relevance of the information to the modification of behaviors 
required for reaching Specified teaching objectives.. 

Different applications of CATTS have been field 
tested and demonstrated in a number of comrnmity inservice 
training sites during the past school year. Space does not 
permit a detailed description of each project, but I shall 
attempt to offer a number of brief examples which illustrate 
sme of our more recent applications of CATTS in inservice 
training contexts. * 

^ ^ The Hammond school project . One persisting problem in 

t^ training of teachers is sens it izing^ them to the n^eds of 
sotiometric isolates in their classrooms and developing their 
skill? in minimizing the social rejection of such children. 
The Hammond Project (Ballard § Semmel, in preparation) is 
designed to use CATTS in training teachers to be more effective 
in meeting the needs of isolate children in their classrooms. 
An observation system was constructed by modifying the Indiana 
Pupil Participation Schedule (Semmel § Meyers, 1973). The 
system focuses on pupil and teacher "behaviors related to social 
participation of pupils in the classroom. Observers used a 
portable Touch-Tone (TT) telephone, which was interfaced from 
the classroom through direct telephone line to the computer 
facility at CITH (approximately 250 miles away) . Observers 
entered eight classrooms each day.^ plugged the TT telephone into 
a common extension phone receptacle installed in each of the 
respective classrooms, dialed in to the computer, and proceeded 
to /code the interactive behaviors of the teacher and target 
child during reading and mathematics lessons. The coding signals 
were instantaneously received, summarized, analyzed, and printed 
out at the Center. Printouts were then transmitted 'back 
the school through the use of Xerox Teleropier and telephone. 
Feedback was in the hands o£ each teachefwithin a ten-minul;? 
period. ' 

ft 

Preliminary analysis of the results of this input -process 
product study are very promising. Significant changes occurred 
in tlie sociometric status of the target children from the pre- to 
the posttest period. We are currently in the process of relating 
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teacher and pupil process data to the outcome variable • If 
our subsequent analysis o£ the data meets our expectations, 
we feel tliat a feasible and effective paradigm for the delivery 
of inservice |:e^cher training, while the teacher is wdrking 
with pupils in her* own classroom, will have been demonstrated. 

Inservice training paradigms using delayed CATTS ' • 
feedback , ^Two field research and demonstration projects have 
been conducted which used the capabilities^ of CATTS in conjunction 
with specified training needs of school personnel. The 
Cuyahoga County Project (Pisarchick, Sitko, § Semmel, in prepara- 
tion) was aijned at improving supervisory effectiveness through 
the use of an observation system and objective feedback of 
teaching performaiT&e to teachers of EMR pupils. The target 
population included 20 intermediate- level teachers and three 
special education supervisors. Observation data was collected 
using DATAMYTES, which are small portable units with Touch-Tone 
telephone button pads. Observers enter "observation codes 
through the buttons, which transmit a signal to a portable 
cassette magnetic tape recorder carrier by the oberver-* supervisor. 
The cassettes are sent to CITH, where they are .replayed through 
a Converter coupler which transforms the audio signals into 
a serially transmitted digital signal --which is then processed 
into computer storage. The data are then analyzed and stimmary 
printouts are returned to the supervisjlrs by mail. 

^ The BOCES, New York Project (Brown § Seninel,*in prepara- 
tion) was an inse^rvice training program which assisted "teams 
of special educators' in studying the interactive processes 
between staff and selected problem pupils. The school program 
is conducted in a self-contained special education faciMty for 
retarded and emotionally disturbed fchildren who are not othgr- 
wise served in the pupil schools. The school, \s organized'on 
an open classroom plan. Two hundred-twenty students (K-12); 42 
teachers, psychologists, and social workers, and 8 administrators 
were involved in the project. DATAMYTES were used for^the 
collection of observation data by the school persftnner, who 
developed an^ observation system to meet their sp|ecj.fic interests. . 
Wjhile CITH staff assisted in the design of the observation system 
and training of coders, the specif icafeLon of system categories . 
and definitions and the data collection process were the respon- 
sibility of the' entire school staff. 

Specification of the parameters of feedback was a 
collective decision o£ the BOCES staff. Cassette tapes were 
forwarded to CITH for processing and summary printouts, as pre- 
scribed by the staff, then were returned to the school in New York. 
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Feedback ranged from spefific beKavior rates on. categories of 
both teacher^^id^rgetrpupil behavior?, to detailed information 
on the child's use of deferent instructional areas within the 
*open classroom environment, to threV-stage sequential chains of 
interactive behaviors with peers and teachers, Jeams met 
periodically to study^e feedback information ^d establish 
common team goals based upon the analysis of the printouts. One 
particularly interesting outcome of this project was the 
staff's consensu$ that the delay of feedback wAs prohibiting 
the maximim use of the information obtained. The administration 
has subsequently applied; for and received federal funds to 
install an linmediate^, feedback CATTS^system within their scjiool 
district, using an existing computer facility. In the past, 
the school district's cdnq)uter had been used primarily for 
clerical and administrative purposes, ^ 

New Directions in Teacher Education Research 

Like so many groups interested in the teaching process, 
A we have followed the primrose 'path conceptualizing teaching 
. the handicapped as the acquisition of a repertoire of relevant 
teacher competencies. Hence, most of our earlier efforts 
in teacher training using the CATTS system have involved ^ 
training teachers to acquire specific teaching skills andV 
feeding back the results of thei^^fforts to use ^h^se 
behaviors with handicapped children. Such a model has led 
us to training^ procedures wl^ich were^ buttressed by a numljer 
of iwerbaliz^^ assumptions of questionable validity. For » 
exmnpje, jteaqhers If^ere typically instructed to increase their 
high l«vel qi^stioning behavior when working with the EMR 
child. It was re^ohed that the use of binary questions 
leads to limited pupil verbal responses. Hence, for the EMR 
who tends to display :^stric ted verbal ability, teachers have 
J a natural proclivity to reduce cognitive demands --thus per.-^ 
\ petuating the limited verbal and cognitive, abilities of suqh 
Children- -and assuring a self-fulfilling prophecy. Therl' / 
are a number of teaching skills which we rationalized as 
appropriate for use with handicapped pupils. However, such 
skills invariably have been assessed, using increased rate 
or duration as a^riterion of successes, and have been 
fostered without atie concern for appropriate antecedent inter- 
active parttems. Hence, we have too .frequently told our 
students to get in tlaere and increase their use of high leyeJj 
questions, or use more probing techniques^, or use more positive 
reiriforcemeat, and the like. We have behaVed as though we 
, knew what good teaclling is- -as though there is a repertoire 
of teaching skills which is universally accepted and invariably 
effective with handicapped pupils. It is most l^ely that 
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fundamental to teaching is, as Hunt (1971) has noted, the 
matching of pedagogical alternatives to specific antecedent , ' 
pupil behaviors- -and that such matching varies from situation 
to- situation, from child to child, and is depend^t upon a 
complex of interacting teacher variables. TeachS^ is more 
than acquii»ing a repertoire of teaching techniques --it is 
more than the maximizing of one class of teaching behaviors 
and the minimizing of another with a specific class of x 
children. Teaching is more than simply matching appropriate 
teaching behaviors to immediate pupil behavioral antecedents, 
the simple two-stage Markovian chain is probably a^poor 
model when applied to the most appropriate pupil-teacher 
interactions (Collet 3 Semmel, 1971)., ■ 

I do not mean to negate the importance of assisting 
potential teachers in acquiring a repertoire of teaching 
skills. This is obviously necessary- -but not sufficient. ; 
Teachers must also learn to discriminate the state of a 
classroom or pupil at a given point in time during an 
educational interchange- -and to select the teaching behaviors 
or patterns which are the most likely to be useful, given 
the teacher's assessment of the most probable current 
states of pupils or classes. This is for mQ=^ essence 
of teaching; it has been referred to by Shavelsoh (.1974) 
as the basic teaching eompetenev- - decision -making . I 
feel that the skill is analogous in its application in 
the classroom to the moment -by -moment induction of attributes 
by treatment interactions. It is somewhat akin to teacher 
flexibility--^ factor noted by Rosenshirie u971) and others 
to be one of %he few teacher competencies correlated with 
pupil academic growth.' 



We must develop paradigms for trying teachers to 
r-^ flex- -to become morei effective in matching their techniques 
A Wj^e states of pufJils and classrooms on a moment-by-moment ^ 
^feSi To my knowledge w t|?ere is currently no systematic 
.litethod for training tedpm's^'in interactive dec is ion -making 
•skills. We are however, encouraged by attempts to build 
formal models which employ Baysian constructs and heuristic 
strategies to conceptualize the necessary steps for teacher 
decison-making in interactive settings (Shavelson, 1974). 

A related area of concern is assumption that 
classical closed-look feedback models^re the most efficient 
cybernetic paradigms- for the, skill. training of teachers. Should 
teachers be^feactors or predictors? I have recently become 
interested in throughput predictive models which may have 
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partidulax-promise in one iitork OJaccia, 1973)^, Fox years , , . 
coimercial bakeries tised elosedrioop models |:o inoi(;itor the • 
progress of baking products.. Each piece of bread was tested 
, a? it left the oven on the conveyer belt. If . a burned loaf 
wisn detected, a message was instantaneously rel^^ through 
a thermostat, and oven temperature was adjusted accordingly. 
The problem was that commercial bakers found that they lost 
too many loalves of bread using this configuration, , Now bread 
is monitored from the moment it enters the oven. Readings 
are relayed to a computer which calculates the probable outcome 
and adjusts oven temperature prior to that outcome* The 
result is a zero loss of loaves of bread. Given our current 
ability to analyze complex chains of interactive behaviors 
in classrooms using multiple observation systems simultaneously, 
yie may soon be in a position to make relatively accurate 
predictions of the consecpjences of maintaining # given complex 
pattern of pj^il- teacher interactions.. Such information could 
be transmitted to teachers in training, who would be expected 
to alter -the protess to avoid an undesirable outcome--or^o 
maintain^l^prdcess the probability of a desired 

event. IVe are currently pursuing the feasibility of [using ^ 
such a model with our CATTS system. Normative da£a tvqm 
Project PRIME, have promise foi/ providing the basic data from 
which algorithms might be devploped for the cognitive and 
affective domains when working with midiy handicapped pupils. 
Where we caiaiently present teachers with summaries of whSf 
has already transpired and expect them to alter their behavior 
as a function of receiving such feedback, a throughput model 
furnishers the teacher with a prediction of what is most likely 
to occur if she persists in a process. Such information may 
prove more effective in training teachers to assess the probable 
effects of their behaviors- -thus inducing, through training, a 
predicting or anticipating skill. 



Concluding Comments 

In this presentation I have attempted to outline some 
examples of the application of systematic" observation in research 
and training in special education, A major point of emphasis 
was that there is frequently great di^arity between verbal 
descriptions of teaching and that which is directly observed. . 
This is most relevant when we wish to relate input variables 
and pedagogical interventions to the growth of handicapped pupils 
It is unlikely that meaningful relationships between teaching 
and ptipil growth can even be established if we^ persist in 
defining special education programs by alluding to the training 
credentials of personnel or iinplied treatments subsumed under 
the rubric of a vaguely defined administrative arrangement. 
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The use of systematic ofiservation in resfeaxch on teaGhing in 
special education Has promise £or:^harpening our xmderstandings x 
o£ those process variables which are most relevant to pupil 
growth. / ^ / V ^ 

Direct observation and feedbaclc, used in conjunction 
with computer technology, may permit the development of 
effective and e'f:^icierit mservice and preservice performance- 
based teacher. training programs. There is much research and 
development work to Be done before a cost-effective CATTS 
systeiftcan be made generally .available. However, 1 believe ^ 
that we havie come a long way toward de^monstrating the feasibility 
of developing such a training system for. special educators, 

' although we lack a program. We do not yet know which testing ^ . 
behaviors are casually related to the growth of handicapped 
pupils. It is' doubtful that we as special educators could 
get any closer than regulsir educators to achieving a consensus 
o'f who is and who is not a go6d teacher. Hence, while we must 

t continue to train teachers, in the absence of ah .empirical 
literature to guide a PBTE program development, it appears 
necessary to at least establish a set of publicly and clearly 
defined objectives representing the skills we hypothesize 
that teachers need* If we establish, such objectives and 
are filling to submit our training efforts to dbjective 
evaluation, than we can at least establish the effectiveness 
of our training techniques. Upon evidence thdt trainees ^, 
attain skill objectives and transfer them to their work with 
handicapped pupils, we. may progress more. rapidly toward 
validating the effects of teiaching through the growt]i of 
Handicapped pupils. Teacher training, in special education can 
progress only when the two issues of training effectiveness 
and validation of teaching effects are systematically pursued. 
Effective training programs are those that demonstrate tM 
acquisition and transfer of teaching skills in accordance^ 
with predetermined traiiiing objectives. Validating teaching 
skills involves the demonstration that such skills are related 
to pupil growth criteria. Systematic observation techniques 
are central to meeting both of these goals in special education 

* research. • 
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Models for Research and Development in 
Perfoimanc^e Based Education * 

Frederick NfcDonald 
Educational Testing Service 

I'd like to correct that record slightly* I'm not \ 
responsible for the Accountability System in New York but for \ 
the design of the plan that they have adopted;^:- v ' , 

Actually, that plan (this is, kind of a diversion from 
what I^m going to say)* is an interesting example of cooperation 
among a very diverse group of people^ , The Conmittee on Account-__/ 
.ability was composed of representatives of the Union, Parents 
^Associations aB4 various groups. They had to agree on a plan, 
and they did, ariU our part was the technical contribution. 
T110r^'s'an enormous amount of goodwill in the city at that level 
to riiake that ^s tern work.. In mdny respects it's a good example 
of the kind of cooperation you can ^et among people if you 
work fairly hard at it. Im' not a believer that you^just get 
people together and good things happen."* ^^y experience is you 
get people together and terrible things happen I There was one 
instance , where a group of people essentially anatagonistic to 
each other worked out their differences very well. And part 
of the problem in this whole competency- based movement is the fact 
that we're always interacting ^ith peojjle that we're not used 
to interacting vdth, and it's beginning to get to me to be 
honest with you. I'm getting kind of ticky- tacky having to 
interact idth school superintendents, teachers, union represen- 
tatives, college professors, etc* I*m losing my sense of 
identity - I think, 'H^ho am I?'^ - because I have to try to 
mediate differences of |K)ints of view on what constitutes com- 
petence among people who differ in their po.ints of view. 

Let me tell you a little bit about the National 
Cornnission and what it's up to. .Right now we are not doing any 
work in special education but that's not by choice, that's 
because the opportunity hasn't presented itself, and part of 
what we do does depend upon either generating an opportunity 
or somebody knocking on our door saying we'd like to work 
with you. The Cornnission was set up to solve the problem that 
of course you've already solved. Since you know what constitutes 
con^etence, most of what I'm going to say is irrelevant. 

About three or four years ago a foundation took an 
interest in something we proposed to them* I'm a little seri-"". 
sitive about mentioning the name of the foundation these days, 
since a member of the family is up for a prqijii^ position in 

*This manuscript was edited b(y the CBTEP staff -Projn a tape 
recording made at the time o£ presentation. Blackboard 
illustratiOTS have not been included. 
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the government. But It was the Rockefeller Brothers Furiii and 
they gave us money really to do a feasibility study. These 
days everybody seems to be getting into competency based 
education and we said maybe there's some way to get people 
to work together - that pie in the^sky approach to 
human nature. But people were doing things and if you could 
get some type of basic organization to do much of the research 
and development work you might actually increase the efficiency 
of what was being done. It was Jthat simple an idea. So we ^ 
talked to many people who at that time were identified with 
the competency based movement, held a number of relatively 
small meetings, talked to everybody we could talk to ^ said 
what does this whole business need. And it was obvious that 
there were plenty of people who were promoting the ideas, 
there were a nximber of people trying to build programs, there^ 
were obviously states seriously considering it and working 
up plans to implement some form of competency based certifica- 
tion. The big thing that kept coming up again and again was 
'TVhat shall be the criteria by which you will certify people in 
terms of competence?"^ 

I happened to know at the time what the position paper 
of the UFT committee on this was going to be: that they 
would essentially support research and development on competence, 
^but that they weren't about to accept certifying .or re-certifying 
people in terms of competence. That attitude obviously was a 
pervasive attitude. You talked to people in schools of education. 
YouM say to them: *'How are you going about defining your program?'' 
'TVell we'd get the faculty together and we'd talk about what 
constitutes competence and we'd agree." And having been a faculty 
member a good number of years I had an intuitive understanding of 
what probably came out of those agreements. That,. is, if you had 
a faculty that couldn't agree nothing came out of it- If you had 
a faculty that could agree something came out of it, and in every 
faculty there's always sub-groups, as we say euphemistically, who 
couldn't possibly agree no matter wh^t happened. There were places 
like Houston which obviously had got a program going betause the 
Dean started it. They started with a Dean who was comnitted to 
the notion of competency based, who went out and hired faculty and 
did other things that deans do to bring most of the staff along. 
The University of Toledo had something going, Weber State had . 
-something going, but the number of instances were small and all 
were particularly vuli^erable to the charge of substantiating what 
it is they were doing. 

That's the real difficulty with competency based of 
course. As soon as you say what constitutes a competence you are . 
bound to arouse controversy. So what we proposed to the Rockefeller 
Brothers Fund was that we create a national conmissiOn whose sole 
purpose was to generate a aational research and development effort 
to study teaching competence or teaching effectiveness, and they 
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agreed to provide what constitutes tJie a<iiimistxative support 
for cheating that coiMnission*^ • . 

The^iConinission is made Up of ^ diversity of people. 
By design roughly half of them are professional researchers*, 
people in universities* or research centers of one kind or 
another. The other half is made up o£ two kinds of people: 
people who represent education professioiAlly, ^like deans , 
union leaders, and school sijperintendents, state siperintendents 
and so on. The other half are made tip of people who represent ' 
the public, such as legislators, governors, etc.* 1*11 mention 
some of th^ names in the latter category: Governor €yme of 
New Jersey is on the Ccmmssion^ Senator Pell, a senator from 
Rhode Island who happens to be the senior Democrat on the eSenate 
Education Committee, Congressman Quie, who most educators know 
about because of his great interest in education. I think he*s 
the ranking Republican member on the House Education Committee 
which is more Democrat than Republican obviously. But he's the 
senior Republican on there, I believe. And Congressman Thompson, 
who happens to be my Congressman. That^s not why he was chosen 
however, but he is a New Jersey Congressman, and a senior Democrat 
on tne House Education Comraittee. 

So the people who represent the public ate in one 
sense people who are close to policy decisions about education 
and they do come from states and areas where there has been con- 
sistent interest in competency based education. Or they themselves 
have been interested in problems associated with competency based 
education. Albert Shanker is a member of the Commission; "^he NBA 
is represented by David Darling, who's, I forget off the tdp of my 
head what his official position is, but he used 'to be the editor 
of the Journal of Teacher Education and so on, but he is the 
official NBA representative. The si^jerintenderit bf the Minneapolis 
public schools -is a monber, John Davis. John Porter, the superin- 
tendent of instruction in the state (if Michigan is a nrember, and 
so on down the line. Several deans, the Dean of the University of - 
Texas, the Dean of what used to be called San Francisco State, now 
CaliJEomia State University of San Francisco, A. Hilliard. The 
Vice-President of Ford International who uspd to be Dean at Portland 
State is there. Then the researchers are a mixture of people who 
have done research on teaching such as Nat Gage who most people 
remember as the editor of the first handbook on teaching. And 
people who have never been in this field, but who are very competent 
-investigators. Anne Anastasi from Fofdham, past President of APA, 
and anybody who's ever studied individual differences has propbably 
read her book, "Psychological Testing". She represents sOTiebody who 
has great research skill and competence and knowledge and experience 
but has never been particularly interested in this field. And 
there's a reasoiv for bringing that kind of person in, . and that is 
to broaden the point of view about research methodology and research 
strategy, by capitalizing on the experience of people who- have— 
itforked in other fields and other problems. 



50 



Well, its an interesting group. ^ We're meeting next 
week, really the first time that the full group has ever been 
together. We've had two meetings before that were kind of 
quickie-like meetings, that is we said come if yoii can: if you 
"don't make the first.one, come to the second one. At 6ne 
point we originally thought of separating what we Vief erred to 
as the socio-political educational types from the itesearch types. 
And the only instance in which the people of the fipd ever gave 
what I would regard as something resembling A directive was 
on that particular point. They saidy "Look, one of the tilings . 
we l^ked about your proposal, was the fact that you mixed the 
researchers and the people who have to deal with the practical 
problems and the public policy problems. Please keep them 
together." So we have, and it has led to very interestiiig kinds 
of discussions. If you're familiar with John Porter, if you're 
knowledgeable or eVen heard about what John Porter's trying to 
do in Michigan you have a state superintendent vflio is very anxious 
to get along with improving the cjuality of education. And he 
said so, very reasonably and forthrightly. And that kind of ' 
influence in a group of people who usually tend to be overcautious 
researchers, is a very healthy influence. And I don't know what 
the upshot of this is going to be but obviously ,! expect to have 
some successful outcome. But it's an inter esgting group *and re- 
presents the kind of mix that ought to be involved in all of 
these different groups. , 

The Commission is broken down into two comnittees: 
One a public pqliq^ior public liason committee that has Pell and 
people like that on it and then the research committee. But even 
in composing those two committees we've put reseatchers on what, 
in effect, is a public policy commfttee and we've put people 
interested in public policy issues on the research committee. 
There's no question about what our go^l.is, ,first to do good re- 
search, but to do it in such a way that it makes a difference in 
how our institutions make decisions about policy. And that's 
what we're up to. Naturally since what we do is spQnd most of 
our time planning a research program, by* and large we've been • 
left alone by everybody else because they say, "Oh, you -go ahead 
and do .that thing you know, we're out doing the real thing." And, 
so far we haven't generated much flak. But undoubtedly we will 
take some positions on matters like accountability and what 
constitutes competence. One of the first meetings, was a long 
discussion on what you mean by teaching performance: What's in, 
what's not in. Harry Broudy, who has not been regarded as a 
friend pf the competency based movment by some people, is a member 
of the commission. He brought the. point up, and we thoroughly 
debated it. 

l^ow the thing that makes the Coninission's research ^ 
program distinctive is that we have said competence must be 
defined in teiTn5 of effects on students. Most people have been 



avoiding that issue, usually in two ways. Thsy say,, "That's too 
•hard to meastrre. To establish the links between the two is 
difficult and threatening, therefore avoid it." Or, "There are 
so many factors affecting student perf^mance that you'll never 
be able to partial out the influence (Sf teachers." That latter 
point I regard as the more critical oni^because essentially it's 
a council of despair. If that were literally true there's no way 
you can make any claims to status or necessity in the teaching 
profession. If you can't deiftonstrate that you as a teadier have 
some kind of effect, so what? Given the general antagonism 
of the public to increasing teachers' salaries, that kind of view 
just feeds into- the hostility to' the professions' attempts to, 
increase the rewards for being a teacher and increase status. The 
other thing , we arc definitely living in an era when a significant 
nuniber of people,' a better way to say it is that a number of 
significant people, h^ve come to the conclusion that the effects 
of schooling are relatively minor and the effects of teachers within 
schooling systems are still more minor. If yoii'll permit that 
butchering of the canparative. And I hear this, fairly reguOarly. 
Can you really demonstrate whether teachers maflce a difference'? 
After all, we know what Jencks has reviewed and so on and so (on. 
And there's a sort of general apathy, indifference, and disbelief 
about whether anything a teacher does or a school does makes any 
difference whatsoever. / 

Nj^w, you might very well 4ay, "Why do-^^oib^e a different 
point of view?" Well ^he reasons are very simple. I don't think 
any of the research that is used to support the position that I 
just described has in fact been very good research. The Coleman 
study for example has only one measure, well they had two kinds 
of measures of teachers, one of teacher aptitude, which was a 
short verbal aptitude test, which I don't usually refer to since 
ETS built it, but it was a vocabularly list.- That's basically what 
it was. A standardized verbal aptitude measure with one of the 
COTiponents in it. Now I'm sure you're aware of the faq,t that 
scores on that happen to correlate .rather significantly with 
pupil performance scores. And economists who are interested in 
educational planning have latched onto that particular "conclusion". 
Because it gives a way of selecti^» teachers that would make the 
system much more efficient. The only other thing that the Coleman 
study produced on teachers was a list of characteristics, back- 
ground characteristics and so 6n. There were no' observations of . 
teacher performance. Everybody knows what the difficulty in the 
Coleman study is, significant as it was. But it's that kind of 
research methodology out of which stveeping and unsubstantiated 
conclusions about the effects of teaching performance come. I 
never thought 1-i4>^e#'i:6^that verb at the end. So our position is 
that we think we can do a better job of doing this research. And 
what I'm -going to do is talk a little bit about the overall research 
design, but this, remember, is in the context of relating teacher 
performance to student effects. Vlhat I'd like to do is lay out 
two basic methodologies that we use as strategies and what we 
expect to find \iheA we do that. 




X Anyone familiar with the design for accountability will 
recognize that some o£ the ideas that we developed there have sub- 
sequently been built into this research strategy. We conduct two 
kinds of activities simultaneously^ One is a set of experiments 
ajid the other'^is a field s^dy type of operation arid I'm going 
•to describe^ the field study^^t}^ of op^i?gl:iori first. The 
minimum requirement to conducjt the type of field study that we ' re 
talking about, is to have two pleasures of pupil perfoxmance at two 
points in time. Now I'm not saying anything about what the ^upil 
measures should be, that is open, in fact the more open it is the 
better. We have jiist finished a study in California whei^e the 
two criterion variables were in the two areas of reading and 
. mathematics. So we measure reading performance in the beginning 
of the year and reading performance at the end o"f the yes^. Now- - 
this is a side point that 1 will refer to veiy quickly. You say, . 
"Why didn't you me'i^sure^ more often?" That was really a fimctioh 
o£ what we were permitted to do in those circifl|istajice?. But the 
absolute minimum is that you get two perf ormmces : one at the 
beginning of a point in tiflfe and the other at the end. ' Now what 
you're trying to do in this field study is find teachers who^are• 
more effective. And what is meant by more effective here is the 
statistical definition of effective. For every child you haVe 
two g^s of scores and what you do is regress Spring scores on 
Fall scores. And when you do th^t you end up e.g., here we'll 
talk about a teacher's class now. Here's a class and this score 
representf where they were in the Fall and this score represents where 
they were in the Spring. So, I'll make up some numbers. Say that 
their mean perfoiiiiance on the measure of reading comprehension 'was 
a score of ten. And their mean performance in the Spi^g was a 
score of Jrwenty. Now for every teacher in the study you would have 
that kind of information. You do a regression anaiy^s* and that 
mSkns you've got points all over the place ,*^eadi of those points 
represents coordinates of. those two mean scores. Here's aomebo^yr-^ 
e.g., who started out with a relatively hi^ initial score bup^^ ' 
whose subsequent score d(j>e5n-^t look veiy high. And you s^liow 
could that happen. Did.it happen is what you want to find out 
first of all. Now when you do this regression analysis you, end 
up by being able^to draw a line through all those points and in 
essence what that line means, for those of you who had regression 
analysis in your training, essentially that line is a post -prediction 
device. This point on the line represents where- you'd expect classes 
which began here to end up, given/all the dp-ta that you've got. 
That's in essence why you draw this line. It's a line of best fit 
around all these data. Now, what's interesting about all this are 
t^e people Whose perform^ances depart markedly from that line. For 
example, if somebody falls up here, what in effect seems to be 
going on? What seert^ to be going/ on is that somebody who started 
at this level and you would predict would end up there in fact 
has ended up way up there . Now what you're looking for are those 
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people, that sub-set o£ people # whose scores, whose class o£ score;s ^ 
depart markedly ttcm that regression line. Thafs the first step 
in the strategy. This is a htmt.and sea^rch procedure. I am trying 
to find somebody who is "worth Tooking at". - ^ 

f Now in the intervening period between the Fall and the 
Spring, what you do is observe teachers })y actual classroom ' . 
obsd-rvation, live observation; the kind of. thing Bob Soar Vfas 
talking about yesterday. You go into classrooms and you .observe. 
And now what you do is take these observation data and try to 
find those factors or facets or characteristics . of these teachers 
that distinguish «iem from the othfe.rs. These data become a set of 
hypotheses about effective teaching perfoimance. O.K. So let 
me recapitulate very briefly. You have started with a criterion 
of pupil effectiveness in general. It's-a gain criterion. . What 
stddents have gained the most, what,groups have gained the most, 
and you then identify teachers where that gain is more that you _^ 
would expect in the ordinary course of events. I'm not talking 
about absolute scores now, I'm talking about the differences 
from thi&iiije. ^I'm not talking about grade equivalents or any 
of that'-d^rt of thing. Having identified these teachers you then 
look at'-any information you have on those teachers that wbuld enable 
you tp say they do this and the others don't do that. 

Now, you know, it could turn out, and when I go back 
r'm going to find out whether it turned out that way, there may ^ 
not be anybody up here. The California scores, 'just looking at 
the sdores from Sprjng to Fall), they've dropped. Now that doesn't 
mean you. won't, find people like this but a bead of sweat emerges 
on the brow when you recognize a substantial number of the kids 
scores are lower in the'Spri?ig than they were in the Fall. And you • 
pan account for that by a variety of factors. So it's conceivable 
we won't find anybody. But there's no other way to look. So i. 
you use this strategy because this is one way "df looking. It's 
the only shovel you've got with which to dig a ditch or maybe a grave. 

Now what we're doing, as a matter of fact it's siipposed 
to be finished Friday, we do this regression analysis across all 
teachers, and we also do the regression analysis within class so 
that lean look at any class (There are 97 classes.). I can look 
at any class on the regression line, and these marks here, now, . 
instead of being mean scores are the scores of individual children. 
And one of the first things I 'm going to look for is the variation 
in the slope of that line. If I find a teacher where the line 
slopes this way. that teacher's having negative effects or something 
is having negative effects on the performance there. You predict 
backwards in that, class. Sp, we're rightat the point now where 
within a matter of two to three weekSsieJal b9 able to say whether 
we found anything. ' ' lA 
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The real criticism o£ most previous work on 'observation 
should be that you look at only teachers behaviors, you don't also 
look at student behaviors. We have a systam, a behavioral recording 
system, in which the observer sits in* the classroom and literally 
writes down everything that the Children do and that the teacher 
does with resp^ct^td the students. Now i£ the clasg were large 
this would be almost an immanageable task so we developed a procedure 
for who to look at in the class. The beginning o£ the year we 
asked each teacher to rank order every child in the class in terms 
o£ how they wquld expect the child to do in reading and in math. We 
also used as part o£ this test battery a standardized achievement 
test just as an anchor point -because the other things that we were 
developing were criterion reference or domain reference tests. And 
somebody always want3-to know how did they really do, where "rdally 
do" means what was their grade equivalent. ^So we gave the California 
achievement tests. Now we're able for every child in the class to 
look at his teacher's prediction of hew he's going to do vdiich is 
her expectations of how he or she is going to do and how in fact 
they were doing. And we sorted ,i^l the children in eadi class, went 
down the list in each class, anS^ere's expectations and here's 
actual peT-formance. We did it at the beginning of the year, and 
again at the end of the year . The teacher says the child will do 
well, and in fact he is a good reader, or she is. The teacher says 
poor reader, in 'fact he is doing well. The teacher has a high 
expectation and there's a low perfonnance. The teacher has a low 
expectation and there's a low performance.- We sorted every child 
in the class into those cells fudging' a little on people who were 
right in the middle. And we took out the middle because they didn'*t. 
vary enough. The teachers predicted they'd do average and they did 
average. So we looked at the extremes and we sampled two children 
out of each of these cells, one boy arid one girl^ and the mix 
represented the ethnic composition of the class. Those were the 
targeted children that were observed. * 

So when the observer went into a classroom what the 
observer would do, ^he would sit down and record everything that ^ 
those children were dbing over the entire day, and everything J:he 
1:each^ did with respect to these children. So what*we have is a 
manageable technology for observatign. At the same timfe, we're 
gathering data that's interesting. Because if the teacher doesn't 
interact yWith these two kids, in this cell, it's riot very likely 
she's going to be interacting with the four other ones that were 
in that cell either. That's sampling again, there's always risk ' 
in sampling but these classes are sufficiently/ small that a 
sample of ei^t in some ca^es is half the class. And it's always 
cabout a third of the class . So you '-re really gett ing a lot of 
information. ^ Secondly, you'rfe getting extensive information^ 
that is, you 'get everything that is done by these eight pupils and 
the teacher with respect to them;' . ^ ^ . 



Well that's one observation system. The other observa- 
tion system tracks the teacher, follows everything the teacher does ' 
and then what the child does in response tp what she does. We've 
iised two observation systemsv, We had abojA 400 days o£ observation. 
Some teachers were observed as many ks eight times in a relatively 
short time. ' ■ - . 

\,Well, those details are irrelevant oth^r than to say 
that kind 'of . a technology is what we're vising in the Commission re- 
search project. We have a lot of experience with those two, systems 
and we're vising them, first of all, because they give lis an endtmous 
amount of information. In that one behavioral recording system 
that I described we have something like 22-jOOO sheets of observation, 
which is a lot. Now there is a lexicon for poding that material. 
The other system Was developed by intensive observation of reading 
teadiers over a petiM of time , and then was developed into a 
category system. And it's called RANDS, which is not the name . . 

of an Egyptian king, but it means Reading and Mathematics Observa- 
tion-System. The other system, the first one, is called APPLE, . 
and none of us can remember what the acronym stands for ^ not &^en 
the person who designed the system. We just can't recall it fot 
some reason. ' 

The strategy here is to use the field study as a device 
for generating hypotheses about effective performance. It also 
becomes an arena which you can develop your skill in construct- 
ing assessment systems. And it makfes, from our, point of view, 
very little difference what the criterion is. The commission is 
not saying what public education should be up to. It's accepting^ 
what people ^ho are responsible for public education agree it ought • 
to be up to. So in the case o£ Califomia^or example, the California 
commission on teacher licensing and preparation said we want to look 
at those skills related to producing effects in reading and mathematics. 
That included reading, decoding skills, comprehension, application, 
and attitudes toward reading. And the state had laid out a whole^ 
series of objectives over a period of time that people, have worked 
up and agreed "^on and all we did was sort those objectives intor 
those four categories and then proceeded to build measures of the 
different types of objectives. And in general, what we do, in any 
area that we work, is find an area where people are interested 
in particular kinds of pupil outcomes or sets* of pupil outcomes. 

Now we have an overall model of the data that we gather 
all the time.** The critical thing in making an analysis of this 
kind is that you want to know the relationship between teacher 
performance, student behavior and ultimately student Olitcome. 
The distinction between student- outcon\e and student behavior is 
a simple one. Student behavior is what you can see the child 
doing as part of the instructional process . . Mainly that means 
looking at what he's doing in the classroom'. In principle it 
could include observing him 'in other kinds of' contexts. It just 
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happens to be difficult to do that sometimes. But student outcome 
is a measure of 'the child's achievement of these significant goals 
of education in his stage in, life, or the level of education he is 
at. Teacher performance is what the teacher does. What you're 
trying to do is to identify how much of a variance in student 
outcome can be attributed to variances in teacher performance. 
That's what the whole research is about. ' 

In order to do tfiat you must partial out the effect of 
other factors on student outcome. So we gathered a lot of data 
on student characteristics. In fact we sent home a questionnaire * 
Cyou know, pin it to them, watch and make* sure they don't throw 
it away, As they go out* the door, call up the parents, get themi ' 
back, translate it into Spanish) in whic^ we asked a number of . 
questions about the child and so on. And incidentally whenever - 
we do this stuff we have groups of people who screen it. We 
don't send out a parent questionnaire without having parents look 
at the thing. Now the other thing that we have picked up measures 
on are student aptitudes. We use a generalized aptitude test 
and anything else we can build into it. We've used in the 
California study a measure of student cognitive atyles. And 
^actually we would like to enrich that battery considerably. 

There are practical problems in testing children, as 
you know. One of th©ni is , if you gather everything you want to 
know you exhaust the children beyond belief. So* you have to be 
prudent in how much you do. But we try to get in as much as 
we can. 

Now, in getting bgck to the teacher performance we look 
at teachers' knowledge, e.g. we gave a decoding test to see if 
teachers haa decoding skills in reading, and it turned out they -did. 
We built a diagnostic test in which children read passages on a 
videotape, and they made errors and the teachers were supposed to 
, identiify the particular reading errors that they Were making, and 
then say what they'd do about it. There was muchrmore of a spread 
of performance on that diagnostic measure than there was on the 
decoding test. Irr other words, teachers can decode (that group of 
teachers) but they're not equally effective in diagnosing somebody 
else's errors. Which. is a very useful thing to know, because it 
tells you where training is probably needed. The knowledge battery 
also included knowledge of teaching of reading methods. It included 
a mathematics test on the reasonable grounds that if teacihers don't 
know elementary school mathematics they can't teach it to the children. 
So that was a complex of factors. 

We have been building a teacher aptitude battery which 
has eighteen different measures of generalized aptitude. The idea 
is that when you're talking about aptitudes you're talking about 
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information processing characteristics of the person. And what we 
were looking ^at is what kinds information processing characteris- 
tics a teacher would carry arjaimd with him wherever hfe would go, 
that how those would affecft the'perfoimaiice. And it covered such 
things as memory tests , ability to store information quickly, ^ ' 
measures of cr6^ativityt, flexibility and things of that kind. There 
were reasoning '^ests in there, and the logic of putting reasoning 
tests in tiiere V^s thaty in teaching reading comprehension you are 
" essentially teac^ing^ a kind of reasj^ping process . Thfere are two 
kinds of comprehfensicm itoughly, literal conprehension and infer- 
ential comprehension. 'i^ In inferential con^Drehension you have to 
go beyond the data that's a<?tually there. You know, with literal / 
comprehension you say "Who killed Cockrobin?" and you say whoever 
killed Cock Robin. And it's in the passage. In inferential com- 
prehension you have to draw conclusions on the basis of the data 
that^ou're given. 

I don't know if Bob Soar's data has"^ held up in his 
later analysis, but in his earliest report there was a ^relationship 
between certain kinds of teaching performances and whether children 
improved in abstract reasoning skills. The teaching performances 
all involve such things as posing questions and encouraging the 
child to generate answers on his own, to formulate hypotheses and so 
on. That sounds like the kinds of things you'd probably have to 
do if y^u wanted to get a kid to reason about a passage. It's hard 
to visualize how you'd help hion learn inferential reasoning skills ^ 
without putting him in an instructional environment in which he has 
to attempt conclusions, evaluate his conclusions against the data 
and so on. So, what we were curious about was what the^ status of 
the teachers' aptitudes would be in this respect. If they 'Weren't 
very good" at inferential reasoning, it strikes us as unlikely that 
they would be do nig much of those performances that required them 
to help the child with his inferential reasoning. We also had a. 
measure of teacher cognitive style. Without my elaborating you can 
seen why we had the student cognitive style measure and the teacher 
cognitive style measure. 

We also looked at school characteristics; had a long inter- 
view with the principal, and we 3tudied administrative climate. We 
had some data on teacher characteristics - standard background information. 
We did an administrative climate study in which we asked the principal 
to talk about how he handled things and we asked the teachers to talk 
about how he handled things. 

Now when you have a set of factors like^this, one of 
the interesting problems to think about is causality. What most people 
do is a multiple regression study in which they continually partial 
out the relative influence of these different factors then they end 
up saying teacher performance accounts for only 5% of the variance. 
We're doing something like that but a little more complicated. 
We're doing es-sentially a structural model analysis. Now what a 
structural model analysis requires you to do^ is to make decisions 

58 ' 

O . -49- 



ERIC 



about causality, the hypotheses really, and then ixse the techniques 
that are involved in doing structural analysis to test your model' ^ 
which strikes jne as a much more enlightened way of going about 
doing research then what we usually do. 

Now J 'm going to draw some arrows here. Whenever I draw 
an arrow it mean^sUiat A affects B in the true causal sense, that 
I can account' for thi$ by looking at that. I'm hesitating ajt 
administrative climate because I really don't have either the theory 
' or data to know what it effects. So I'm going to guess and do this. 
We assume that these factors affect directly teacher performance 
and these these factors are related to teacher characteristics. 
Now school characteristics, when you* re not sure what's horse and 
what's cart, what you do is talk about it as a correlation - you 
draw a curved line with arrows at both ends. And the statistical 
analysis simply takes into account that correlational relationship 
but it doesn't enable you to partial out causality. I would say, 
e.g. that you really have to think about there being a correlation 
' ' between school characteristics and student aptitude. But its hard 

to figure out what's horse and what's cart. So, is it the aptitude 
that makes the student characteristics or is it the school charac- 
ter istics, or do the school characteristics to som6 extent affect 
student aptitudes. Some prograjns may very definitely affect 
aptitudes. You think of aptitudes more broadly than the usual 
verbal aptitude stuff. For example, visual information processing 
skills is an aptitude, I think, which intuitively is highly 
sensitive to environmental factors, I mean instructional factors. 
So that if you get a lot of trailing in visual information processing, 
you have visual information processing skills. If you don't get " 
a lot of training in that, you don't have them. So you give a 
spatial visualization test, it may very well be a function of the 
extent , to which the instructional program empha^sizes visualizing 
things in space. You people are into learning disabilities so you're 
familiar with the study of perception and that sort of thing. That's 
the kind of thing that we try to lump into this student aptitude 
battery and then we're saying if you talk about school characteristics 
you caxi talk about programs. Do you have perceptual training programs 
in the school? If you do that would affect aptitudes. Also, however, 
you can look at school characteristics as the function of the kinds 
of children that are sent to that school. So schools will differ in 
aptitude levels purely as a function of how they're sorted into 
schools. 

So what you do is you make up these arrows- I'm leaving 
out a few arrows here, and you do a structural model analysis, path 
analysis. And when you get numbers on this, what you get, essentially 
is a vector, and the number is the magnitude of the vector and the 
diagram tells you the direction. And you can trace indirect "effects. 
For example, I can trace the effects of teacher characteristics,, 
teacher knowledge, teacher performance, student behavior and student 
outcome. There is a path through there, and I can tell you how much 
each exponent in that path contributes, and which component contributes 
the most to student outcomes. That is exactly .wha,t we're looking for. 

erJc '59 



Now structural analysis is. a ccawnon sti^tistical technique. 
It's use<i mainly by economist^; it's jiist a variation on multiple 
regression analysis. But it's the sl^le that goes with it that 
counts. That is, you've go to layout this path diagram before 
you do the formal analysis. 'When you do multiple regression analysis 
you do all the calculating and. then you try to account for the ^ 
results ex post facto. You say "tiiis relates to that, and this 
relates to that, and this looks like that, and so on. Now what we 
do, is we use all these data that we've constructed in the field 
study to do this type of structural analysis and to identify those 
characteristics of teachers which distinguish; them from other 
teachers iii terms of whether they're outliers or not. ^ i- 

That's one big strand. Every study we build has a field 
study coinponent in it, because we need a procedure by which we 
^continually generate new hypotheses about effective teaching per- 
formance. And if we do studies in a variety of places as we're 
plannihg to do, you get a lot of variation in school characteristics 
and administrative climate and student aptitude and so on. Now 
we're trying to design studies in New York in conjunction with 
the State Department and the New York State, United Teachers that / 
will probably be concentrated in two places: New York City and 
one upstate area. I sometimes say ray idea of iipstate New York 
is the Bronx but I've been disabused to that. We've been talking^ 
about Syracuse, but we're looking for one of those large metropolitan 
areas that has a mixture of middle sized urban, suburban and rural 
constituency in schools. • 

Where we work is really a function of a lot 'of things . 
First of all the three of us have to agree on areas and' one of 
the sources of agreement, one of the bases for agreement is 
whether there's a kind of labor peace there. There's no fitting 
between the board and teachers. Because you could never conduct a 
§tudy in that area. The other is you've got to be able to pull 
together a research staff in that area. Tho6e are two general 
requirements. The other places we're working are Detroit and Lansing, 
Michigan, Toledo, Ohio and we hope eventually to be doing and some 
more work in C^ifomia in conjunction with the State Department and 
the Califotnia Teachers' Association. And eventually in Texas and 
Miami, Florida. So the projects will include most of the major 
urban areas, in the United States (New York, Detroit, San Francisco, 
Los Angeles, Miami, Houston or Dallas) and, presumably, whatever 
comes out of that research will have some credibility. If the same 
data show Up in New York as show in Texas, the Texans won't be 
able to. say, "Well that's New York." Nor will the New Yorker be 
able to say, "ThatJs Texas." Or if there are differences we'll be 
able to relate it to something meaningful other than the fact that 
we live in different parts of the country. / 

I have neglected to say that most of this research is 
conducted vdth experienced teachers. That is, the people we're 
looking at are people teaching in real classrooms* And the logic 
of that is that if you can find people who are effective -there it 



-51- 

60 



makes sense to "look at them. We pick experienced teachers because 
they've adapted to the system in some way, that is, they can work 
in classes and with kids, and so on. Now in the process of course, 
it is conceivable that we've missed scHnebody who is an outstanding 
teacher and dropped out of the system after one year because he or 
she can't stand the system. 

.1 really don't know how big a risk it Is. I'm sensitive - 
to it because when I was a professor in the West one of the hardest 
persons we had to place was an intern, an exceptionally "bright 
young woman, dedicated to teaching, but she was of her own mind. 
We were over in Castro Valley, California which was a center of 
John Birch activities and she walked into the principal's office 
and she had jio chance of a job the minute she stepped inside the 
door* She was a very bright young woman, very adept, skillful and 
worked well with children. But she's hard to place, and she was . 
reaching the point where she was so fed up with the system she 
was not longer willing to try to find positions in it. So we may 
be missing people like that who, if we could vrettch would have very 
unusual characteristics. As I say, I don't know how big a risk 
that is. ^ 

On the other hiind, the research^has a real atmosphere 
of reality about it. We're looking at real children under the 
ordinary circumstances of schooling and teachers who are actually 
working with those children. Now, once we get a set of hypotheses 
we then begin the second strand. So one strand is this field study 
down here, and that's going on continuously. ,The other strand is 
a series of experiments, and that's the experimental strand. That's 
the hypothesis testing. The field study is the hypothesis generating. 
So this is hypothesis generating, and this part up here .is hypothesis 
testing. Now what we do in that component is take a group of teachers 
and essentially put them through the following kind of sequence of 
events . 

You first have a training ^erience. Then they move 
into actually using the skills on which they have been trained 
in a classroom durihg which we observe than, this is classroom i 
performance and at the end of this sequence of time we measure 
pupil outcome. We learn a lot with this particular strategy. 

First of all, in the training part we try to use, or 
will try to use, different training methods so that we get a 
test of the viability and the effectiveness of different training 
procedures. Do you use models? Do you use feedback? What com- 
bination 6f models and feedback? . How much is direct instruction? 
Who mediates the instruction? We'd like to bring in some admin^ 
istrators to do some of the training here to find out whether, ' 
you know, the general bifelief that they ought to be more involved 
In the improvement of instruction has any possible effects on 
achievement. But there is great freedom at this point to devise 
different types of training methods and try them out and find out 
what's effective. But the training is on specific skills. 




You then observe the teacher wsioig the skills to see if 
the training ca:rries over into the classroom and then you look at 
the effect of the specific skill. 1!his coimection between these 
two pieces of data, the pupil outccjjne and the observation tells 
yjou whether if they do A does it produee B. ^d that's . the real 
test. Thi? part tells you soraething about tfansfer of training 
from the ttaiiiing' situation to the classroom sittiation and it 
Vlso tells you something about effective |;f^ining methods. 

Now those experiments are relatively small scal^, 
we'll be using fractional factorial 'designs which permit us to 
use a relatively small number of teachers but still get very 
powerful set of experimental data. They will be relatively 
short term in length,, i.e., we usually say the t,rainiD[g is the' 
equivalent of one week intensive -training and this is the equivalent 
of t^tfo or three weeks observation. That varies depending upon what 
the skill is or whether you can treat a skill as independent or 
you have to do a couple of skills at a time. And, in general, we 
plan the study so we do two or three experiments a year, or four. 
There's a limited number of days you can work in schools because 
of holidays and everything else that goes on. Sotthis requires . 
a continuous run of activity. So there. aren't many blocks when ! 
they're all there for long periods of time. So you can do about ; 
four experiments a year. * ' ' 

Now all these studies are the. combination of the field 
and experimental studies; that? is, the field study keeps feeding ■ 
in ideas for skills %o be teamed in the experiments.. Experiments, 
will probably give us ideas ior other things to look at in the 
field. These studies are continuous over time, i,e. they all are 
planned to last about five years and stay with the sample of teacjliers Y 
during that five year period. So it has some of the advantages 
of a ease study. You're looking at the same people year after 
yeat after year and you accumulate an enormous amoUnt of data -on 
those teachers. And that becomes, iji effect, a case study of 
those teachers. The overall strategy has the advantages of a 
lot of field data to generate hypotheses. It has the advantages ; 
of doing experiments to test hypotheses. It is longitudinal in 
character, it uses live observations of people, and itdnas the ' 
advantage of gathering so much data on teachers in schools that ; 
in fact you can do intensive studies of* individual teachers 
over time and yoa-gather a lot of information about the schools 
themselves . \. 

So that's the overall framework. I hesitate to say 
we'll work with anybody because that makes me sound like one of 
the major service institutions and I'm not thinking of the police 
and the fire department. But we are interested in getting a 
range of projects in terms of the following characteristics: 
diversity of pupil outcomes, and diversity of settings, and 
we prefer scope and breadth rather than small chunks of research. 
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^ I think we've conceptualized the strategy which if we 
caji bring if off is a vast iinprovanent%pon what we've been doing. 
We're not doing simple laboratory studies pne-at a tiAie. So if 
there is anything to be found here I think we've conceptualized 
the strjciegy that's likely to find it. 

We are working with the relevant groi:5)s, the State ' 
Department of Education in New York and NYSUT. The proposal'!? 
been presented to the people in the State Department, up above. 
Vince Gazzetta> and Mike sit in on all the meetings, Chuck Sgntelli, 
Jeannie Kemble sit in on all the meetings. We work as a team irt 
designing the study* It's been* presented to the executive 
conmittee of NYSUT and, if j)rou read the New York Teacher, Tom 
Hobart, in an issue or two, referred to it. The idea behind all 
of that i& that if these results are to have any meaning the 
people who are going to' be affected by them have to be participants 
in the process from the beginning. 

As a matter of fact we're taking an entirely different 
position with^ respect to the role of teachers on research. We have 
to state our official position: that teachers are co-researchers 
in this process. Now the standard objection is they don't know 
any research. That objection ife not a very good one. 

First of all, te^cji^rs as a group are intelligent, people. 
What we're doing here is not very fancy. The mechanical ^statistical . 
part of it you need special training for, but the rest of it 
reasonable people can think about. And, speaking from the viewpoint 
of the research community and the academic community, it seems to 
me it's in our best interest* to involve teachers because the way 
you learn research is by doing it. And if you want the public, 
which in, this case is teachers/and so on, to support educational 
research, they really ought 1p know i^/hat ^ it ' s doing and be involved 
in it and understand it. We've pretty much taken that official 
position. We've worked' through the organizations, and we involve 
teachers at all levels. 

The New York Project will have teacher review panels 
and administrative review panels. They look at materials because 
we don't want to be a bunch of ^researchers that go out and ask 
people questions that offend them, and probably don't have any 
utility as far ideas are concerned anyway. So that's our 
style. It is a^omewhat different style in research and it has 
real difficulties. When 1 insist, for example, that teachers 
be co-participants on these projects, not in New York but outside 
of New York, I create trjDuble. And I'm regarded as a trouble-maker. 
You know, you're from the Eastern establishment, vnu're qominq 
in here telling us what to do.... I say, I'm not telling you what to do. 
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You don't have to Work vdth us, and we'don't have to mrX vdth 
you. But we're not going to work on this project with people 
being sort of first class citizens and second class citizens. 
Practically, you learn an awful lot from teadiers that are 
involved; We've; worked very closely With people that are e)t- 
perienced classroom teachers and when you're dealing with this 
kind of research you know they know what's gding on and what's 
do-able and vdiat i^lt do-able. Certainly scijie person like me, 
living off in' the lalds of Princeton, doesn't have that kind of 
faftiiliarity. 

Well, that's what the Commission is all about, and I 
should Veally stop talking, and I apologize for talking so long. 

, /Thank you. 
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ASSBSSMEOT IN CBTE: SOME TEACHER CONCERNS AND CONTRIBUriONS 



James G. W?ird 
New York ^tate United Teachers 



I am here this afternoon to attempt to articulate some 
of the concerns classroom teachers have vath assessment in 
teacher education programs and, to .highlight what I feel are 
some contributions teachers can make to the process. I 
would be remiss if I did not make it very clear from the 
beginning of my remarks that this is a role with which I 
feel uncomfortable, Fbr too long all sorts of individuals 
have been speaking for teachers without systematically 
determining the attitudes, perceptions, feelings, values, 
and opinions of classroom teachers I have no desire to add 
to this.. What I say here will be my viewpoint, the view- 
point of an experienced classroom teacher who now works for 
teachers^ trying to help them deal with their own particular 
perceived educational needs and what I say will be based on 
both roles I have filled. . - 

Competenqy based teacher education as a movement and 
-vits a reality in New York State is a phenomenon that I have 
alA^;ays viewed with a critical eye. Teachers in general 
have seen fads and innovative practices come and go over 
the years with little impact on impx;oving the quality of 
education for cliildren. tVhat I am really saying is that in 
addition .to all the specific criticisms leveled at CBTE, 
with which we are all familiar, we teachers, for good reasons 
approach any new programs imposed from above without signifi- 
cant ^practitioner input* with a certain amount of cynicism. 
To state it simply and to the point: I view assessment as 
the most ainportant factor in the future success or failure 
of CBTE in New York State, and I also am convinced that jteachers 
are the one group that is going to make CBTE work if it is 
going to be a success. To not give proper consideration to 
assessment and to teachers will build in automatic failure 
in any teacher education. process or program.^ ^ 

I prefer to look at assessment in the context of an 
Input -Throughiwt-OutCOTes-Iihp Model. In a teacher educa- 
tion program in assessment the supreme test is the impact on 
society or at least on the educational process of the teachers 
that^ha>e gone through the program. The desired impact 
should be articulated in broad goal statements. The .problem 
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inherent is that in education i and similarly in other areas \ 
of social concern, "to conduct a legitimate » impact analysis 
would require collecting data over the entire professional 
lifetime of the teacher. Obviously tljfis will not do / so" we 
must go back lape step in my model and look at Outcomes. 

Measurement of outcomes i^s the assessing of the extent 
to which, the' pre- service. teacheV has r(^ched the objectives 
set for the teacher education program. I believe that, to'^ 
have any utility, these objectives should be eixplicitj public,- 
' and, to the degree possible, quantifiable. A major contribu- 
tion of teachers to CKEg assessment is playing a major and 
significant rqle in the development of these objectives. Lest 
anyone has not already made the connection, the objectives about 
idiich I am speaking are statements of the skills, Imowlcdges, 
and attitudes a pre-service teacher is supposed to acquire as 
a result of the program. 

V Three problems come to mind immediately: 

1. How do you know your objectives are meaningful and 
valid 1^ meet your goals? The answer is simple. You donH 
and probably can't in the short run. Only after years of 
program operation and with sophisticated evaluation devices 
can this question be answered with any degree of objectivity 
and surety. This is why major and significant teacher in- 
volvement in developing objectives is important. If you can't 
scientifically assess the appropriateness of your objectives, 
then let's at least have the people in the field, on the firing 
line, the people most familiar with^the ultimate job to be 
done set the objectives. Reality testing must suffice for the 
time being. 

2. Secondly, developing an assessment program to measure 
movement toward meeting these objectives is no easy or simple 
task. Here I don't offer solutions, but only highlight 
problems and concerns. If the pre-service teacher fails to 
meet X number of objectives, does this indicate a deficiency 

on the part of the pre-service teacher or program deficiencies? 
How do you control for the effect of factors external to the 
program? Or don't you really care? If the assessment process 
• is going to have any degree of cost effectiveness, how do you 
deal with the problem of fair and comprehensive assessment of 
a wide range of objectives? How do you build in a due process 
system to insure that no pre-service teacher is unfairly ideal t 
with? This task is difficult and crucial to program success. 
Hie development of assessment systems are the proper purview of 
skillful competency evaluation research specialists. I 
mi^ht add that few of these presently exist in education, . 
either in the field or in colleges and universities. 
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3. Thirdly, is the problan o£ people with the proper 
^ training and skills to conduct the assessments o£ pre-service 

teachers^ This is going to be a time consuming task. With- 
out dwelling on it at any length, let me say rchat it also ^ 
requires specialization and skills that are ntt now Widely 
foimd. This is too important a component to Be left to ama- 
teurs^ 

Beyond these three problems there are other considerations 
that must be dealt with an4 that areof concern to teachers.' 

The first is a strong warning; using the model I men- 
tioned earlier, outcomes are not impact. Don't unconsciously 
make that confusion I The state of the art in educational re- 
search is that, we cannotn simply n^ke facile connections, assuming 
a causal relationship betwben teacher behaviors and studei\t per- 
formance. I repeat,, the research/is riot there! In fact, this 
is the rationale for a joint research project of the New York 
J|f State United Teachers, the New York State Department of Education, 
and the Educational Testing Sfeivice connejpted National Com- 
mission on Performance Based Education that is now in its 
beginning stages. This research project is heavily classroom 
based and this is an example of how actual, real, in- the- flesh 
classroom teachers can make tmd are willing to make a great 
contribution to solving our dijemma. We need much more work 
and research in identifying /teaching skills, developing and 



using training materials, and iitejien^^strating the effects of 
these skills. Teachers deserve "^d demand greater participa^ 
tion in these processes. 

Another concern stems directly from the monumental task 
to which I have been alluding. Great care must be taken to 
insure that any shortcut/ that leads to greater individual 
program conformity; rath0r than to greater individualization in 
the teacher training process wil not be taken. The tempta- 
tion will be great, but it must be overcome. While paying 
lip service to all th4t is involved in formative evaluation 
01^ assessment and to^^criterion referencing in assessment^ w*1;'^ 
had better not fail to ccnne through. All I am saying is that, 
especially considering the pressures of time and the fru:§trar 
tions of inadequap research, if we take shortcuts, leave out 
essential elements, and design superficial assessment programs, 
we^are taking a /giant step backward in teacher education in 
,this state and teachers have real fears along this line. 

i previously mentioned due process guarantees. Formal- 
ized process0s must be built into every assessment program to 
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insure a fair and equitable recourse for those people idib may 
feel that they were dealt vdth unfairly in the assessment 
process, this is due all candidates as a matter of basic , 
hmnan tight and is particularly a concern of teachers for 
assessment in programs leading to permanent certification. 
Teachers would weiccsne the chance to assist in the develop- 
ment of due process procedures and it is incuinhent upon those 
with management and decision-making responsibilities in CBTE 
to insure that teacher views are solicited^nnd are carefully 
considered.. , • 

I have been discussing what teachers can do, *and I 
should mention what I think they^should not do. I feelf^^ 
strongly, and many teachgr s sha fe my view, th^t cooperating 
teachers should ncrt: be? part of^^y assessment or evaluation 
of pre -service ^eabhers which would be of a suntnative 
nature. In order to maintain the non- threatening, profess- 
^ ional, and helping relationship that must, exist between . 
student teacher and cooperating teacher, the sole respons- 
ibility for evaluation must remain with the college or uni- 
versity personnel. This is even more crucial for field 
experiences in programs leading to peimanent certification. 

Yet another concern of teachers in assessment is in the 
area of the use of assessment data. This infoimation should i 
only^ be used for the purpose for which tKe data is collected 
and strict confidentiality should be maintained. Release of 
this dat^ to potential or current employers perverts .its 
original' purpose aid will lead to rapid dpterioration of any 
kind of cooperative relationship in the asses-sment process. 
Assessment will become a stratf^gic game and education will 
suffer. ^^--^ 

A final word. Local contracts between organized ^ ^ 
teachers and school districts are legal agreements mutually 
developed in a complex and highly political environment. 
Assessment may well bring college people into the schools- - 
at least I hope it would. The contract must be respected in 
every way. Violation by ignorance is no excuse. A word to 
the wise should be more than sufficient on this topic. 

I hope I have given you an idea about the kinds of 
issues and potential problems about which teacher? have con- 
cerns. I hope also that you maybe now have a better feeling 
of ways in which teachers can make iirportant and useful con- 
tributions. ''As emphasized before, assessment i!5 a key to 
the success of CBlE and teachers are the key to assessment 
procedures. Involve teachers, listen to teachers, heed what 
teachers have to say. Solicit true teacher views and don't 
let others speak for teachers. » 
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Gilbert Duken ! 
' Broad Street School - Plattsburgh 

I was pleased to be asked to spekk toyou regarding 
some possible concerns that School Adminis tractors may have 
regarding the many phases of the CBTE Prdgraml It's truly 
a great ^opportunity to be able to look ahead in the endeavor 
to identify potential problems , for too often administrators 
spend more time then they care to in the process of solving 
problems rather than preventing them.' 

As all of us know, problems can bte found everyvdiere 
that we care to look. Often it's advisablgJi3u4gMre them as 
I did at my home yesterday. Just before fwis lea^^g, my 
wife inquired about my lodging arrangements. Her response , 
to^my staying at a Seminary was a strange onq* I've been 
wondering now if maybe hsve a problem .tor I'm not sure she 
believed me. Communications are bad all' over! . . 

Speaking of communications, there's a story that I'd 
like to share with you. I was visiting a neighboring school 
recently and had occasion to overhear a conversation in the 
Faculty Room. Someone had just said, "C-B-T-E, vfliat is it?*' 
Several teachers responded. For example, one teacher offered 
that CBTE meant Creating Botherspme Teaching Experiences . 
Another clearly stated that CBTE really stood for Correcting 
Backward Techniques of Education. Another adde^l that the letters 
were for Curse Brought to Education. Previous to this analysis 
of initials I had been sure I knew that CBTE meant Cbmpetency 
Based Teacher Education, but suddenly I left that faculty 
room wondering just what did it mean to me, and what does it 
mean to my faculty; " 

; It appears that there are a variety of attitudes 
concerning Competency Bashed "Teacher Education and its expected 
inpact on public schools,' teaching, and tJie student. I'm ' 
pleased to say that I'm finding a positive reception to this 
new program, which by consortium design assures input* f I'om the . 
various interested and involved element^. This consortium- team 
is cl^arged with a difficult ta%k: that of identifying appro- 
priate cjHi^tencies and establishing activities by vfliich 
measuremen#may be achieved. Although cotnpetenGy identiifica-^ 
tion, etc. is a significant start, indeed the next step seems 
to me to be the crucial one. The pfesoit small team approach " « 
must now execute a concerted effort to achieve fiill^bloom 
support and^ enthusiasm from all the niany publics in education. 
I stsrori gl>^ beli eve that our CBTE Program ir doomed to failure 



if we overlook our obligation to public relations. We must 
educate all educators regarding this new program and seek 
their positive participation ^d cooperation previous to 
program implementation and previous to discovery o£ antici- 
pated program weaknesses. It is jji^jerative that all be a 
part o£ the force that solves the problems, not creates 
them. Let's not be like the man who met major difficulty 
when putting the watermelon into the refrigerator because he 
avoided initially taking out the beer. cans. 

Indeed, it is. foolish then to have a competency program 
if there is no rbom for it^in the minds of our publics. So 
we ask, how can we present this program so that we eventually 

, achieve enthusiastic and spirited team support? This will 
probably be the most difficult task we have, for apathy and 

' lack of commitment of people to work together continues to be 
the eternal plague of group effort and achievement. Neverthe- 
less, the CBTE program should have a head start toward achiev- ^ 
ing extensive commitment by the very fact that the competencies 
have been conceived, written, and in essence accepted for ini- 
tial operation from all the areas of concern via the consortiimi 
approach. A warning however, if you have not begun this way 
then already you have a significant handicap and have over- 
looked a wealth of contribution necessary for a balance of 
opinions v*iich will surely be encountered throughout the CBTE 

, program. A narrow approach no^^r may very well develop into slim 
participation later and result in opposition which may gener- 
ate problems that might have been avoided. 

The first major concern then is to attempt to achieve total 
involvement, not only^vdth the consortium approach to competency 
writing but especially in educating the many publics plevious to 
program implementation. This public relations effort does not 
have to secure innumerable participants, but, if done correctly, 
it will obtain a following of infotmed, and interested persons 
who a^e ready to supply a positive force of support and peripheral 
assistance. To build such a force, I recommend the following: 

1 . Broad Orientation Meetings be made available for all 
teaching personnel. Such meetings should clearly pre- 
sent the many positive aspects of the CBTE program and 
indicate direct professional^ and personal gains to be 
expected or desired from such a program. Attempt to 
redirect all the negative concerns in such a way that soon 
those \A\o were asking why, become the ones to ask why 
no't. Try to schedule these meetings at individual buildings 
during convenient times for the teacher. 
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2. Specific Insexvice Orf^ntation Meetings or Itorkshops 
seem necessary for those Special Education Teachers 
who are considering the role of cooperating Teacher 
and therefore will have a more direct^ responsibility 
with the CBTE program and student teachers* Two . 
major concerns that may be dealt with via this Ift: 
service Workshop approach are: ^ 

A. The cooperating teacher m^ have been previously 
^ r prevented by time, distance, or some other reason 
I ^ from participating in recent professional courses 

and therefore be, or at least feel, inadequate at 
providing the necessary experiences for the student 
teacher. Ffopefully these available workshops, 
meetings, courses, etc. will alleviate this cbn- 
cem. 

B. Secondly, possibly separate from, or in conjunc- 
tion with these meetings on up-date teachijig methods, 
is the opportunity for cooperating?^ teachers to be- 
come informed of the method of management or imple- 

^ mentation of the CBTE program. The cooperating 

teacher must understand his role and responsibil- 
ities and he must accept both, before participa- 
• tion can occur which is positive and productive. 
This is not only important for the cooperating 
teacher's success, but it will eliminate prob- 
leihs for Institutions of Higher Learning if the 
process is previously understood and acceptable 
to those who- are most directly involved. 

3. My thirdireconsnendation for building a CBTE force is to 
have the selection of participants based on merit,, which 
can only be achieved if the program is attractive and 
valued'^ by its potential participants. To make this 
possible, a systan must be devised which significantly 
acknowledges service rendered. The most obvious way to 
do this is for the cooperating teacher to be financially 
reimbursed for these services. If this is not feasible, . 
then consideration might be given to the waiver reward; 
or perhaps college facilities be made available which 
would not otherwise occur. 

If indeed these cooperating teacher services are truly 
valued then our institutions must do something to properly 
indicate it. Although teachers are motivated in many differ-^ 
ent ways and will often give their best far longer than the 
best is given them, they nevertheless have a legitimate com- 
plaint when their use in the teacher-training program of any 
and all institutions is abused. For indeed, they are not paid 
to be cooperating teachers and deserve some motivation beyond 



profession^ dedication for becariing so direttly involved and j 
responsible for the success of the teacher-training program. 

Therefore, it is vital that we obtain positive and 
optimum participation previous to the C&TE program implemen- 
tation, Wfeiy I urge those responsible to attr^lctiv€^Iy and mean- 
ingfully and conveniently motivate and encourage totfiii under- 
standing and involvement from as mdny pubXics as possible from 
the very beginning. 

The second concern which I see is a weakness that may be 
the offspring of the desired consortiumapprb^ich: having many\ 
individuslls a pdrt of the continuing GBTE prtice^s. The consort- 
ium vehicle which I believe is necessary for ultimate success 
may also carry with it an undermining element if We are not 
careful. This element of destruction, as I see it^ is the team 
that has no designate captain, or the vehicle, if you will, that 
has no driver. Everyvrfiere that the CBTE program exists there . 
must be a recognized leader who's indisputable reponsiblity it 
is for the success of this program, and the democratic adminis- 
tration of it. Cooperating teachers do not want to tell the In- 
stitutions of Higher Learning how to run its business, Hpwever 
they do expect openness to suggestions and participation-re- 
quests of them which are meaningful and appropriate to the goals 
of the program. Surely we want to avoid the type of situation 
v^ich was encountered one day on a street in New York Gity, It 
seems this man was standing on a comer minding his own business, 
when a bum comes up to him and snarls, "Gimme a buck!!' "^The man 
iwas slightly taken aback by the comment and stood right up to 
him and said, 'TVhat do you mean, a buck?** 

The bum said, 'Tou heard me, buddy!" 

This time the man raised his voice and said, "Look! If 
you asked me for a dime, or a quarter, or a half dollar even, 
but a buck! You must be out of your mind!" , 

Then the bum said, "Get this, buddy, you can cough up 
the buck or not, that's up to you, but don't tell me how to 
run my business!" 

Although we are all privileged, and pleased to be a part 
of the GBTE team, we must be sure we know who our leader is. 
Let us not make^^he-Tnistake of telling the Institutions of 
Teacher Education how to run their business, and let's hope 
they will not make the mistake of not seeking our valued 
advise and assistance. 

However, knowing who the leader is, is not enough! The 
leader must also know who he/she is, and meet the responsibil'- 
ities of this role. The failure to do this will ultimately 
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result in program deterioration. As an administrator^ I worry 
about inadequate leadership, for I want all my programs and 
my teachers to meet with maximum success. Avoiding problems 
is much more pleasant all the, way around, aiii therefore, IM 
like to suggest three considerations of leadership which I'm . 
sure are obvious to us all but which unfortunately continue 
to be the agents of failure . They are': 

1. Communication must be established v^iich is continual 
and convenient. 

2. The management model and the program gpals must be 
clearly understood and organized, yet flexible 
enough to meet individual needs and problems. 

3. The leadership must provide appropriate and consis- 
tent evaluation activities that concern all areas of 
the CBTE program and provide avenues for on-going 
channels of improvement. 

The leadership role is a key-role and although the 
Colleges can share some of the duties of leadership they can not 
legally give the responsibility for it away. We have every 
right to expect our Institutions of Higher Learning to provide 
inspired and energetic leaders. For, without a doubt, program 
achievement in any undertaking, directly mirror the quality 
of leadership shown. 

A third concern I have is not as obvious as the othpr tw 
and does not effect the success or failure of the CBTE program.' 
Rather it is the pupils or the student teacher viiich could 
possibly be adversely affected; perhaps receiving an incomplete 
curriculum due to pressures placed on the cooperating teaclier 
and student teacher in the competency based program,. 

In the past, the student teacher has often had to "fit the 
mold" of the cooperating teacher *s classroom, carrying on in her 
traditions, with limited regard to specific teacher education 
goals. Although I see no dfkstic changes in this situation, I 
nevertheless anticipate some modification in teaching methods 
by the cooperating teacher to meet the demanding student teacher 
competendy programs. This will obviously have its advantages, 
but it could have its disadvantages also. For example, ^we may 
find we have a few teachers who suddenly make selections of 
their curriculum activities based solely on their ability to 
measure competency levels of student teaching. Or, worse still, 
they may decide to determine curriculum, as well as classroom 
activities that primarily are designed to measure competencies. 
Although I don't see this concern to be an extensive one, I 
nevertheless feel that administrators should be alert to the 
possibility. I was particularly remind^ of this area of con- 
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cem last week when I was reading my daughter's ski magazine. 
It* contained a short story which Glearly illustrated an^ exag- 
gerated effect to" a child who had been over taught in one skill 
to the detriment of learning another. The stoty took place on 
a ski slope an Wisconsin. The youngsters were havjig^ the tme 
of their lives and skiing with an amazing degree of skill. 
One little tyke who couldn't have come higher tjian my knee, 
strode intp the warming house, accompanied by-ms not -much 
bigger sister. ;vhen the little fellow was asked by one of the 
grownups if he Could really ski, his only answer was a toothy 
grin. " , 

"Of course he can ski. Ma'am", his sister volunteered, 

"but he can't talk yet." 

Although this story over simplifies iny point, I neverthe- 
less believe there is reason to be cautious and ccrtiCemed that 
cooperating teachers and student teachers not neglect the total 
curriculum and skill areas of academics, in the procedure of 
meeting isolated competency levels. Every effort mist be made 
by all involved to choose student teacher activities and' ex- 
periences which meet the needs of the pupil first, and secondly 
the goals of the student teacher. 

The school Administrator is the' one responsible for 
assuring that this concern does not materialize. It seems 
that the easiest way to do this is before the problems exist; 
therefore, selection of good cooperating teachers must be 
achieved. As mentioned earlier, selection for this program 
should be based on merit , for good teachers would know how to 
improvd the program for pupils with student teachers, which is 
as it should be. 

Earlier I indicated the need for our program leaders to 
have management models and evaluation techniques which were or- 
ganized, understandable, and appropriate. Again I mention this, 
primarily because I see the cooperating teacher as a central 
figure in the evaluation process" of the student teacher. My 
fourth concern then is tnat this process Is reasonable and prac- 
tical to accomplish. I am fearful that the evaluation and man- 
agement processes can become confusing and/or troublesome. 
Should this occur, it will either severely detract from the 
teacher's class and planning time, or he/she will become dis- 
couraged and not be willing to act in the capacity of coopera- 
ting teacher. Either vay we have a problem. Therefore, I hope 
' all efforts will be employed to conceive an appropriate process 
and select effective and efficient tools to implement this pro- 
cess. 
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Equally important is how this evaluation effects the 
student teacher. We want to be sure we "turn them on", not 
"off". As ^foman^4/4Fice^^ Peale said, "Most of us would rather 
be ruin^ by^raise than sa\Md by criticism." This must be 
considered in our methods or reporting and record keeping. No 
student teacher should be abused by our program, fairness iraist 
be an essential ingredient. There's a story of ari alledged 
injustice in evaluation which amuses me. Instead of recording 
something on a critical analysis sheet, in this story it was a 
ship's log. On that day, the" captain entered this brief 
account: J'K&te was drunk today." When the mate became normal, 
he was terribly chagrined an^ angry; he pleaded with the cap- 
tain to strike out the record; he declared that he had hever 
been drunk before, and promised he would never drink again. 
But the captain said, "In this log we always write the exact 
truth;" The next week the mate kept the log, and in it he. 
wrote, "Captain was sober today." 

Evaluatidis can be misleading. Let us be careful to 
* avoid these kinds of problems, as we deteiTiime our evaluation 
process. " 

Still another area of evaluation we must not overlook 
is t]\e need to evaluate the competency itself. We are just 
beginning and there's much to be learned. This evaluation pro- 
cess too must be carefully planned so that we continually grow 
in a constructive, positive and cooperative manner. 

In any event I am eagerly awaiting the implementation 
of the CBTE program with all student teachers. Far beyond the 
many areas of concern v/e may have, we have every reason to 
expect numerous professional gains, far in excess of the anti 
cipated problems, I believe the competencies will not pnXy pre 
pare better teachers, but I believe they will motivate our exis 
ting teachers to improve their programs and methods. Hopefully 
too, this program will help our students who are education 
majors to recognize their teaching abilities and inabilities 
earlier than in the past, allowing them, if need be, to select 
alternative career training curriculums, consistent with their 
talOTts and abilities. We all know that •a doctor buries his 
mistakes, and an architect advises his clients to plant vines, 
but vtfiat do we do with the mistakes of teachers? Or, maybe we 
should ask vtot can we do to avoid employing teachers vtfio-make 
mistakes. Perhaps our CBTE program will provide us with a 
giant step toward this solution. No question about.it, our 
strides are getting longer, and we're on the right rbad to up- 
lifting the quality of teacher-education and teaching. But 
most important, this quality will produce better programs for 
^children, and particularly handicapped children. 
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ODNTRIBtiriONS AND OMERNS OF PUBLIC SCHOOL AIMNISTRATO^S 

. ' ■ ■ : „^ 

Michael Sdlimando 
James E. Allen Learning Center • ^ 

I appreciate the opportunity to Speak about the 
contributions and concerns o£ the school administrator as 
ConlEJetency Based Teacher Education programs begin to ifllfold. 
I Mght add that the word-reactions could also be included 
in describing this administrator's craimentary as spme of my 
remarks this afternoon will be in the form of reactions to , 
several aspects of the CBTE movement as it pertains to Special 
Education. ^ 

One. of the initial concerns and tasks for the school 
administrator will be to provide "some enlightenment and educa- 
tion about the term - Competency Based Teacher Certification . 
Fijr examRLe., does this term imply that teachers whp may be from, 
out-of-^tate or be vetet'an teachers and who have not. been 
through a competency Vased prograni are not competent. Ob- 
viously not, but parents of children in Special Education have 
been dealing with labels f|r many* years. Priorities, in t^jns 
of labels, are developed by parents, individually and in groups. 
Parents in Special Education relate tg labels and place signi- 
ficatKie or meaning, sometimes incorrectly and inaccurately, on 
such labels. It will become a responsibility of the special 
education administrator to educate parents as to the meaning 
of Competency Based Teacher Education and Certification and 

what it does and does not imply. 

■ . ■». 

Veteran teachers may need some guidance and direction 
from the school administrator in acquiring an understanding, 
if not an acceptance, of the tepi CBTE as well as the substance 
of this new developnent. The role of the school administrator 
wit|i reg?ird to the Cooperating Teacher may be the 'area of 
greatest concern to the administrator and the area in which the 
administrator's role is most greatly affected. The significantly- 
different approach to the education and certification of future 
teachers will necessitate a different view of who or what type 
of experienced teacher should be sought for- service as poopera- 
ting teacher *for pre-service teachers. If, in some situations, 
the cooperating teacher was always*a reflection of the adminis- 
trator supervising that teacher that reflection will be shattered 
by the CBTE movement.. The word ORCHESTRATE becomes prominent 
in this regard. More and more it will become the function of 
the school administrator to orchestrate the efforts of the 
, several individuals ori^arties to the preparation of would-be 



teachers within the framework o£ Competency-Bas^d Teacher 
Education. The new and greater involvement o£ the teacher- 
training institution, the local teachers V unit and the coop- 
erating teacher will cause changes in the role^ o£ the school 
administrator wi;th respect to the cooperating teacher; 

Much has been said and written about the status o£ 
teaching as a profession. We in education are >COTpared to 
other professions and fields of service with the conclusion 
very frequently emphasizing those areas in which we fall 
short of the full status of a profession. Literature prepared 
in support of CBTE cites the development of competencies that 
wHl provide the adequate and systemati9Jqniowledge base which 
our '•profession*' so desperately needsy'School administrators 
should welcome, encourage ffiid help any efforts which will pro- 
vide the service of teaching with the fixll status of a profess- 
ion. 

Turning to the identification of the specific competen- 
cies, the acquisition of which will lead to certification in 
areas of Special Education, what is emerging is a group of 
COTipetencies sufficient in breadth and scope to both permit and 
encourage varying objectives in Special Education teaching 
situations. As we know, labels, particularly so-called diag- 
nostic labels, such as mentally retarded, brain- injured, 
emotionally disturbed, etc., are employed principly for admin- 
i^traftive and organizational purposes but certainly do not 
mean that each brain- injured child is a carbon copy of another 
child with a similar malady. As an administrator in Special 
Education, I do not see a diminution of the eclectic approach 
which many of us in Special Education hDld in high regard. 

The nature of the supervisory or evaluation process as 
practiced by the school administrator may undergo some change 
as the number of practicing teachers who have been prepared 
via the CBTE route increases. Because of the complexity of the 
competencies that are being developed within Special Education 
and the resultant, number of possible objectives for a particular 
class or gruop in a given lesson, the Pre -Observation Conference 
will become more and more commonplace. Administrators will find 
it necessary to obtain initial input from the classroom teacher 
in terms of competencies to be dononstrated, objectives to 
be attained, etc., prior to conducting the formal observation. 
The administrator will, in many instances, not be able to walk 
into the classroom without some prior preparation if he is to 
provide tlie classroom teacher with additional support, other 
ideas or means of achieving the same objectives and construc- 
tive criticisms. 
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In caning to a conclusion about the school administrator's 
involvement with Competency Based Teacher Education and Certif- 
ication, I vfould like to cite some of the positive "spin-offs" 
which this movement has had and will continue to have for 
Special Education: ' 

a) Movement to Criterion - Referenced Assessment; 

b) Educdtion courses and the fact that there 'is no 
longer a prescribed set of courses; 

c) Evsauation in terms of student progress, with the 
understanding that the teacher and/or the school 
do not hold all of the influencing factors; 

d) Bnphasis on the end product, the measureable, the 
oljvious and the concrete; 

e) Affective education and the fact that education of 
the affective domain can and should be included in 
Competency Based Teacher Education. 



I Lastly, I would like to caiment about one aspect of CBSp 

which has been mentioned at this conference as well as at other 
forums and in the literature. These concluding remarks relate 
to my earlier cOTinents about the status of teaching as a pro- 
fession. This movement may very well find itself more and more 
in the realm of a labor issue which could result in additiorfal 
problems and differences among staff members in any particular 
school or locale. Should such a development occur, one would 
' hope that the issue would not be over the trivial or mechani- 
cal aspects of CBTE but, rather vrould be whether or not this 
movement will provide the adecpiate, validated knowledge base 
for the teaching service to attain the full status of a pro- 
fession. » ' / 
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COmiBUTIONS AND CONCERNS OF TEACHER EDUCATION STUDENTS 



Karen Cochran 
sue - PLATTSBURGH 



First, I would like to mention that I am presently doing 
my student teaching in an EMR class with eight pupils - three 
girls and five boys. The following points are raised in an at- 
tempt to determine effective procedures ff or assessing student 
teaching experiences. ^ 7/ 

iNhen we talk about competency based teacher Wxcation 
we mean that a future teacher should be competent in the aca- 
demic area of his or her own education: a basic core of aca- 
demic subjects and electives. A series of method courses in 
actual teaching procedures and techniques could be most ef^ 
fectively evaluated by the respective university. 

In the actual student teaching experience, the cooperating 
teacher is an important person, especially with respect to 
assessment. She wjLll help you identify your strengths and 
weaknesses and offer possible solutions to problen^s. A coopera- 
ting teacher is best qualified to a^fc|S^s your c^Snpetencies 
because she is most familiar with your performance in the 
classroom. %fer assessment may be formal or informal. Both 
are valuable to the student teacher since you have the benefit 
of her^experience to guide you. 

Feedback in the classroom is ano/cher'^good vay of assessing 
a student teacher's competencies. Af tfeV. a student teacher has 
tfiught a lesson, it would be beneficial for her to review the 
material to see whit the pupils have gained. 

School administrators are also valuable in assessing 
student teachers' competencies. They actively observe the 
student teacher's interest and attention to performing duties 
within the school situation. They also observe the student 
teacher's attitude and enthusiasm with relation to extra cur- 
ricular activities. 

/ jThe college supervisor acts as liason with the college. 
Hfevisits the student teacher in her classroom in order to 
asreSs what progress is- being made. The more frequent his( 
visits, the more accurate will be the evaluation of progress. 
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So far I have discussed the people /who vrould be injvolved 
in ah assessment of the student teacher's competencies. How 
would these people -go about their evaluation? Infomal meetings 
and discussion groups with the student teacher would be helpfiil - 
These eval^ators attempt to wrk with the student teacher to 
guide her in finding her own solutions to problems or to dis- 
cuss different approaches to problems that the student teacher 
might be having. These people help the student teacher refine 
her techniques and acquire a degree of professionalism and 
respect for her role in the education of children. Constructive 
criticism and praise, where earned, are valuable forms of evalua- 
tion to the student teacher. If the college supervisor and 
cooperating teacher work closely with the student teacher in 
evaluating her teaching tecltniques it would make it a more 
valuable experience for the student teacher. For example, 
perhaps the student teacher could sit down with and conpare her^-» 
own self-eyaluation of a particular lesson vdth her cooperating 
teacher's and college supervisor's. Having the c:|assroc«n pupil^ 
'answer- a questionnaire about wh^t aspects the pupils enjoyed and 
benefited from most could also be valuable to all concerned. 

. » 

Self- evaluation is also an ijnportant part of the assess- 
ment of ccsfnpetencies. The student teaclver would have*iad 
first hand observation opportunities up to this point^ 
Learning from observing and comparing teclmiques observed from 
the student teacher's own situation would also be valid form 
of self -evaluation. Tlie student teacher should use tlie prac- 
tice teaching situation as a trial period for testing out the 
best teaching techniques which would suit the individual's per- 
sonality. The student teacher should attempt to evaluate her 
own rapport and interaction mth the t^pils based on a feeling 
of mutual respect and trust. 




oifTmiBUTIOf^S fim^ CONCERNS OF TEACIIER HXJCATIdN STUDENTS 



\ I have reached a point in the course o£ my student 
teaching vAiere I am aware of several concerns. Qae of the 
most urgent is the direction of certification and training 
for elementary and secondary school, teachers. Since my 
affiliation with the National Teacher Corps began in the 
summer of 1973, I have seen examples which lead me to be- 
lieve teaching is perhaps the mo^ pampered and paranoid of 
tlie professions. 

Du^I hajte heard teachers complain about the unfair treat- 
ment M education by society. ?tost teachers feel that tax- 
payers should allow the schools tQ completely control » their • 
childrens* lives and that the schools are not accountable for 
the product they produce: namely a child wHd is equipped 
to cope in today's world. If educators want more money from 
the taxpayer they will have to show m earning from the monies 
they are already getting. . 

For more than three years I liave been involved in some 
form of CBTE. Some of my experiences have beengrewarding 'apd 
some have not. My major concern, or fear, with the planned 
implementation of certification by the CBTE method, is that 
becailse of the unmllingness and lac>: of cooperation from 
those teachers who could benefit most by this type of training 
or re-training, the new system ma)r be led an^ participated in 
by^many of the very capable teachers who would be effective 
and efficient in v^atever manner they taught their classes. In 
the meantime, the teachers wIk) are debilitating: tlie goals of 
education are permitted to continue tlie same practices. 

V/Jiat is to be done ^dth teachers wlio feel they are doing 
an adequate job of educating" the young, yet v/1k) are not even 
scraping the surface of their minds? If you attempt to forc^ 
the system on them, many will robdl and fight, throvnng those 
wishing' to try it into an even more conplex state o'f doubt. 

I ant also concerned about the possibility of tlie loss 
of academic acMevement in the classroom if tlie entire format 
is going to change to CBTB. It is my feeling that the pre- 
test, post- test method and individual instruction can be of 
benefit to some students while inhibiting others. Does CBTE 
meai/1£haj: we are going to mter down our standards to insure 
the (student of a passing mark in a course, or does It mean 
.that)1^fe will tolerate differential performance criteria? 




Nathan Glasper 
S.U.N.Y. -ALBANY 
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Upper level students who are aware of the philosopliy of 
CBTC are sJiakiiiR their heads and wondering, whv tliey sliould work 
tidce iis Iiard to achieve an "A" wJien tliey could very well take 
it easy and receive a 'T". Some of our better students need 
to feel tliat tlie systcan is not clianfjinn to ma1:e life at sclx)ol • 
easier for teacliers hut to make learning an enjoyable exper- 
ience for all students. I emlirace scHfne of the concepts of 
CBTR, but I will have to witlihold my final judRmont until I 
can be sho^vn it is a trend in tlie right direction. 
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CONTRIBUTIONS AND CONCERNS OF TEAOffiR EDUCATION STUDENTS 

Karen Kbwalchuk 
S.U.C. - PLATTSBURGH V 



Inherent in the Qoncept o£ cprnpetency based teacher , 
educatin is the idea that a clear, concise statement can be 
made of the roles, responsibilities and. functions of a teacher. 
In conjunction with it is the notion thajt teacher educators 
can agree on what the necessary and essential characteristics 
of a teacher are. It also infers that once agreement has been 
made as to what these traits are, these qualities can be effec- 
tively measured and^ evaluated. It also assumes that knowing 
and accepting all this, the teacher -educator will "he willing 
and able to adapt to flexiblQ instructional programs. Jt 
assumes that students preparing for the profession wll, in 
their relative short training period, be able to grasp fully 
and explicitly, ^gxactlr^wh^^ these characteristics are and 
furthermb^be able to adaoris$rate or illustrate such material ' 
through this flexible instructional approachT^^^-- ,. 

^ It a^jpears to lae that there ar^ tod^rma^xy assuraptioi|,s 
being made here. Many qualities df afgopd and COTipet^t teacher 
are acquired through years and .years,^ experience, "Observation 
and. trial and error. Many teachers. .never become good ajM>\^ 
competent, yet the teachers in 'trying- will be asked to illus^ 
trate these necessary cpipetencies in their training periods. 
That idea is very^rightening; to be responsible for demon- 
strating qualities that often take years tjp dev^op. 



What makes competency based teacher education even/more 
frightening is the evaluation process 1 Jt will be the utost 
crucial test of competencies.. Since it is the most crucial 
test, it raises several other questions. How will the e^^lua-;' 
tion information be used? l%eii will the evaluatiotv take place?' 
Upon what wil^ the /evaluation be based?' By wharf will the . 
evaluation be^done? Will tliere be specific periods in- the. pro- 
gram Where 'the students will, know that they do or do not or may 
or may jiot be competent in 'the necessary areas? ^ 

In order to be fair to the students it seems that eVal-- 
uation must be used as ^ process rather! than a product. ^ Since 
some skills or competjencies need time aM experience to ^develop, 
the. students must be allowed to err at /various stages of devel- 
opment. They must be allowed to 'use the e^jaliiation as^*^an aid 
in decision -making and in improvement i rather than have it ^ 
* immediately 'asses,s thpir viorth or m^lt as teachers I . " ' 
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In order to use the' evaluation as a process, as an aid 
in decision -makiiio:, it must be used as an instructive seciuence 
to brins; out the weak areas as vrell as the fetrbng ones. In this 
way the students will have better ideas to what competencies 
or slcills nBed ;£uiflier work- and attention,,^ If it is to be used 
as a process, evaluation mist he dnno continually - day to day, 
week to week, month to month and year to year. 

Obviously the students will have to he evaluated in 
relation to thcr stated objectives i iii relation to the prograr^ 
set up and Li relation to the degr^ of attaiiniient of the re- 
quired coi.TnctencieSv T^ie dej^rec of attairu^ient TOuld also have 
to be in . relation to individual i>rogress, perhaps group progress 
and possibly national norns on degrees of attainncrt of other 
iiiulviduals throughout tlic couiitr)'' at certain periods ip^ the 
program. By coi;iparing these degrees of attainment on a national 
basis, tlie students will Icnow hovr cannetent they are in rela- 
tion to others in the program in general. / % 



In order to be evaluated fairly throughout the program, 
t'le^ students wuld have to be assessed by a numbed of individ- 
uals. One ^individual may be better ablfe to evaluate students on 
specific objectives at certain points iji the program than others. 
Cdisideration should be given to the possibility of individual 
student evaluation on a day-by-day or iveek-by-v/eek level, with 
a final evaluation based upon the compilation of the datum "in' 
toto". This final decision v/ould then be the judgment of the 
worth or merit of the stiidents ias * teacher s. For instancd, it 
dould be the ^college teachers* job to evaluate the students 
on their Jcn6wledg6 of the material to be taught. The school 
supervising teacher could evaluate the students' ability to 
handle the pupils being taught, as itfell as observe student/ ^ 
pupil behavior in the classroom. Evaluation of| competencies 
oh an ongoing basis couli be done tlnrotigh the lase- of audio/visual 
aids, pointing out rates- and degrees pf pyogress. in relation 
to <:ompetencies . The overall changes in the achievement Iev6l5 
of the pupils being taught could be a sendee of evaluation 
.demonstrating effectiveness. Tests similar to the Flanders 
Test could be used by the students themselves to assess their 
abilities to handle andnise pupils questions and responses. 
Scliool administrators could observe and perhaps assess the 
students ability to function within the scii^ol setting in 
relation to other teachers and pupils and even parents. After 
thies^ people have conpiled these types of information the 
teachers unions may assess the students as to their overall, , 
abilities and .competencies as teachers. TTie anonymity of the 
experienced teachers would aid in a fair evaluation of the 
students. It would be this final step that would ultimately 
determine the students' con5)etencies -as teachers, yet by using 
evaluation as a process, a student ivould have a fair, idea as to 
abilities and potentials in the*ariBas of the stated competencies. 
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CONTRIBUTIONS 'AND CONCERNS OF TEACHER EDUCATION STUDENTS 

Margaret Maxwell 
S.U.N.Y. -ALBANY 



yjho is the competent vteacher? What deeds has he or 
she accomplished in. order to\be avyarded such a title, or, more 
accurately, vdiat refoimation has occurred to change a novice 
into a master? What powers have been procured by what person- 
age to bestow suqh an award? These figurative questiohs\iay 
sound too flippant in presenting an interpretation of Conriet- 
ency Based Teacher Education, but, for myself, a Teacher Qorps ^ 
intern in the SUNYA - Schenectady program, I feel as ecjually 
overwhelmed by the prospect of being -evaluated as a competent 
teacher as that novice must have felt. To be consi(tered a 
master of my subject area, a psychologist in my handling of a 
child's personality, a social worker in my relationship with 
that child's coranunity, a lawyer in my defense as a member of a 
school 'comnunity, and a humanist in all of these is an incred- 
ible and frightening manent of a self -actualization. But I 
am being trained and evaluated in such roles and raust face 
that realization. Just how, then, is my own teacher -training 
program in general bringing me to ccanpetencies in these roles? 
How specifically does the SUNYA Competency Based program attempt 
to accon5)lish such a task? More importantly, does it succeed? 
In answering these questions, I will'deal with the following 
points: 1) Are the skills and characteristics of a competent 
teacher accurately defined and 2) Once assvmied to be defined, 
how are they measured or evaluated? 

Let me begin by asking V'Does Competency Based Teacher 
Education define the skills needed by a competent teacher?" By 
this I mean those skills vdiich ^e trade-defined such as the 
carpenter's skill in wDod-shaving>itli^surgeon's skill 'in opera- 
ting, and the writer's skill in choosin|~-t^ appropriate grammar. 
Does CBTE present to its students those skills necessary to func- 
tion in a classroom setting? To this, I must admit yes. After 
16 months, most Teacher Corps interns in the Schenectady School/ 
system can use a film' projector. We dan also use the techni- * 
ques of set, closure, redirecting questions, and brainstorming, 
We^ have,, therefore, acquired the skills that define our trade. 
But what about a second and far more important: definition of our 
teaching profession- --what about the characteristics of a ccnnp- 
etent teatoer? Characteristics differ jfrom skills iii that they 
don't necessarily manifest themselves in physical actions as 
skills do. 'Characteristics Stan from feelings -impressions one 
gives to another; subtlfe inferences tikt can mean so much be- 
tween a student and teacher and can mean the success of a child's 
learning. Does CBTE define these characteristics? ,In my opin- ' 
ion, it does attempt although not as successfully as it does in 
the skill- area. The Teacher Corps -SUNYA 8th cycle proposal states 
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the. following as a chief characteristic of a teacher education 
program: "A System of Self-Analysis and Talk Interpretation must 
be included in Teacher Education.'* (p. 52V I quote: "A 
combination of the professional sensitivitySand the human rela- 
tions ccmiponents vdll permit' development of an y^propriate 
style of humanistic teacher education." Justwhmt does this 
mean to a student -teacher?- What is self -analysis? How is it 
performed? To what* end should I be. analyzing myself -- to be 
a better teacher, social worker, lawyer,, or jperson? And just 
what is humanistic education? Is it a ccimbination of Gestalt 
and Skinner ian philosophies to create Maslow's "Psyc|iology of 
Being? Studying such philosophies does not define the charac- 
teristics of a competent teacher, and here, CBTE falters. One 
can easily define a poet*s skill in using rhythji, but what about 
his characteristic of sensitivity or perceptiveness? The sur- 
geon'sv^skill in removing a cancerous breast may be defined, but 
what afibtft his ability to inform a parent of a child's death? 
Just how does one define a competent teacher's characteristics? 

This leads me to the second area of questionings CBTE: 
measurement and evaluation procedures. How easy it is to give 
each student a cognitive test on the knowledge level regarding 
the year Columbus crossed the Atlantic. The ansv/er has been 
knovmi over years and years of history books. -Certainly, then, 
the measurijig of a student teacher *s knowledge of specific 
skills is also easily accomplished. Performance evaluations in 
the psychomotor domain are also easily done. The filni pro-* 
jector either works or not, indicating whether or not the stu- 
dent knovre how to operate it. Problems arise, however, in the 
affective domain v*iich parallel the same problem CBTE has in 
defining the characteristics of a competent teacher. Take for. 
example the behavioral objective in the affective domain that 
follows: the teacher shall motivate her students to attend a 
human relations workshop. An evaluator is attending this 
teacher's -class to measure the objective. The student's won't 
go. The teacher is judged incompetent in that characteristic^ 
Hopefully, this is a hypothetical situation but for discussions 
sake, I am criticizing the inability of this evaluator to pre- 
assess 1) the type of atmosphere in which the objective is 
being met. For example, there might have been a fight previous 
to this class meeting which has caused the students to "turn 
off". Wbuld an evaluator know this? Should he consider this 
in his evaluation? 2) Have the teacher and evaluator ade- 
quately prepared for such an evaluation? Do each know what' the 
other is trying to accomplish? Oftentimes, while skills may be 
easily defined and measured, characteristics are impossible to 
measure without considering every idiosyncracy in an .educational 
setting the students, the teacher, the weather outside, the 
principal's speech over the intercom. Can this type of consider- 
ation" be made in CBTE? Is there any valid measurement in the 
affective domain? 
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Competency based education fails therefore, in just 
.this area: the creation of a program where student -teachers 
IJ know the characteristics of a competent teacher and 2) 
are accurately assessed in such characteristics. Although the 
affective, domain has been discussed through academic courses such 
as Adolescent Psychology, Educational Psychology, and Bduca- ' 
tional Curriculum and Instruction presenting ideas on charac- 
teristics of a competent teacher, there is not a truly non- 
subjective,r valid tool, for assessing such an area. If the tool 
of measurement is not available, it is inappropriate and un- 
ethical to judge an individual as either competent or incomp- 
etent. This IS not to say that f feel teachers should there- 
fore be competent only 2n the skill area of teaching. Cer- 
tainly the various roles of teacher as masted of his subject 
area,- psychologist, social worker, lawyer, and humanist can- 
not and should not be defined solely in terms of skills. The 
development of our children's own skills and characteristics 
depends too much on those of a teacher. , It is ny- recommenda- 
tion that CBTE. research the definition of a competent teacher in 
the affective^ domain and create a viable evaluation tool. In 
this manner, CBTE could prove to be an honest and effective 
•reformation of novice into master. 
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Worksheets for Consortia Meetings 

Patricia M. Kay 
Baruch College o£ The City University o£ New York 

♦ 

There are four separate times during this conference that 
have been arranged for consortia members to work on plans, problems, 
and solutions in CBTB for the teacher* education programs at their 
institutions. The major purpose of the conference is to support 
the work of consortia by providing a context within which the groups 
might be able to address their own assessment-related problems. 

We have att0[i5)ted to arrange for the general sessions, the 
facilities of the resource roOTi, and, very in^rartantly, the informal 
exchange of ideas among ^colleagues to facilitate consortia meetings. 
The worksheets, that follow provide a structure that is intended to • 
assist groups in conceptualizing the task ahead, identifying po- 
tential problems, and proposing solutions. 

* ' 

IT IS UNLIKELY THAT EVERYOIC WILL AGREE WITH THE STRUCTURE 
PRCfVIDED BY THE WORKSHEETS OR WITH ALL OF THE STATEMENTS MADE. 

J 

If the members of your consortia agree that for some reason, you 
cannot live vdth this structure or make use of it, by all means, 
devise another format for addressing the problems, proposing solutions 
and working out plans. Your facilitator or one of the "roving'' 
facilitators should be able to assist you if you wish. 

It is important to note that this structure is a framework 
and like the CBTC framework, c&i contain various approaches to and 
conceptions of teaching. 

The worksheets are based on a small "model" for building an 
internally valid assessment system. The model carries no iinprimatur 
from any group nor is it meant, in any way, to "dictate" how things 
should be. The model represents an application of ol^fashioned 
test construction principles to a new-fangled problen^^ It is offered 
M the hope that it wrll help consortia in conceptualizing the task 
of developing assessments systems, evaluating their own progress 
and foreseeing and forMtalling potential pitfalls. 

The pages that follow refer to the model that is diagraimied 
on the next page. Please refer to the diagram when you get to each 
of the major sections (Roman I, II, and III). 
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FOR iUKOIHG IMTE^AIL VAllOm IHTO CBTE MEASURES 



In CBTE, assessment addresses the identificatibn of com- 
petencies, the evaluation of training programs, the measurement 
of prospective teachers success in acquiring program ccwnpetencies, 
and research to validate those cQn5)etencies. How well prospective 
teacher's competencies can be measured is a basic question to 
all aspects of an assessment strategy. The model addresses this 
question. 



STAGE I 

Assuming that we do not now have in hand the research 
data to support construct validity of all measures for all programs, 
competency identification may proceed along course conversion, 
task analysis, theoretical derivation, or just plain %est guess" 
routes. However the knowledges, attitudes, and skills for CBTE 
programs are identified, the result is usually a collection of 
competencies that attiei!5)ts to define a particular conception of 
teaching or approach to teaching. ^ 

A first task iiji Building an internally valid set of coti- 
I>etency measures is to compare the array of knowledges, skills, 
and attitudes that have been identified against the conception of 
teaching that is generally agreed' to be the program goal or philosophy. 
(Judgment 1 on the model.) 

If competencies have been derived from a particular the- 
oretical position or approach to teaching, chances are greatev- 
that the set of competencies will be conceptually unified and, 
in fact, reflect a particular conception of teaching than if a ' 
course conversion route has been taken. A course conversion approach 
to competency identification may very well be a legitimate way of 
proceeding - it's iust a bit more difficult to get an overall 
picture of the resiilt. If that route has been taken, it might be 
wise to digress from assessment development to attempt to organize 
an ovQi^all conceptual scheme. ' 

In thinking about this stage of the process, what problems 
do you see that your consortium faces? . 
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What particular constraints must you operate under? 



What unique . resources do you have for this stage? 



STAGE II - 

Cranpetencies may be measured at various levels or in various 
contexts. Simply stated, they can be measured at a pre-per£ormance 
level - the familiar college classroom methods, in nd^o teaching, 
mini/- teaching and other sijrailatidn or laboratory coiftexts, or in 
real public school classrooms.* , ' 

Since hardly any program can assess all skill's at all levels 
some crucial decisions need to be made about what to measure in 
which context and still remain consistent with the original conception 
of teaching (Judgment 2 on the model) . Thfe first part of this task 
IS to outline the indicators of each competency in all three contexts. 

At this point, it might be helpful to choose one competency 
statement (either from the General Catalog of Teaching Skills or 
from your own prograjn) to* see how tljis procedure unfolds and as a 
basi^ for further discussion. 



Competency ,- 



Brainstorm the possible indicators of that competency in the 3 contexts: 



a 



*^*Jch of this section is based on notions presented by Richard 
•L. Turner in The Power of Competency Based Teacher Educat ion. Benjamin 
Ce^P? ^^-^yn ^ ^acon, 1972 ahd in A G6neral Catalog of T eaching 
Skills, Multisfate Consortium on Perfonnahce-llased Teache? Education" and 
Eeaa^ship Training Institute for Protocol Materials ^""^^Lion ajia 
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College Classroom Lab-Simulation Public School Classroom 
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' The second part of this stage is most iiiiportant to the internal 
validity question. The array of competency indicators in all three . 
. contexts must be compared to the original conception of teaching to 
insure, again, that the measures will accuratisly reflect the conception. 



In choosing which competencies shall he measured in which 
contexts, CBTE program developers may make decisions influenced more 
by the resources available for assessment th^ by the original con- 
ception. For example, if no laboratory or simulation facilities are 
available or if there is limited access to public school classrooms, 
what effect will decisions to assess competencies at t^e college 
classrocin level have on the original conception? Or, jumping ahead 
slightly^ into Stage III, what if, ideally, assessment of a particular 
competencies within the conception should take place in the public 
school classroom but the only .way of measuring the indicators seems 
terribly unreliable? | 

In thinking about both parts of Stage II, what are the potential 
problems your consortium might face? 



What particular constraints must you operate under? 




a 



What unique resources do you have for this stage? 



STAGE III 

Assuming that indicators for each competency in each context 
have been identified and that decisions have been made about which 
context (s) will become the main measurement contexts for each com- • 
petency,^the actual measures need to be collected or constructed. 
There are two parts to this stage also; developijig specifications 
for the measures md then the actual construction and/or collection 
of them. 

To develop specifications for measures, each of the following 
questions needs to be addressed for each set of competency indicators 
in each setting. It might be useful to keep the competency you used 
in Stage II in mind as you progress through the questions. 

a. What task(s) can we give the prospective teacher that 
will elici€=> the indicators? 



B. How will the responses to the task be recorded? (Counted 
and tallied, video or audio recorded? not recorded?) 
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c. How will the presence or absence of competency in- 
dicators be scored? (percentage? frequency tally? 
norm referenced? criterion referenced?)* 



What reliability of measurement Iwill be considered 
acceptable? 



The measures for which specifications can be drawn can then 
be compared to the whole set of indicators and judgments made as to 



the degree of fi^. 

With specifications for the measures in hand, i:he ^ctual task 
of collecting or constructing them can be Jione- 

Whether to collect or construct mjay be a purely practical 
matter. First, are they available for collection? If not, what re-, 
sources are necessary for constructing them? ^ 




^ In thinking about this stage of/ the process, what problems 
do you see that your consortium might face? y 



What are the constraints imder which you might be addressing 
^ Stage III? • ^ 
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, What are the unique resources on which you can. depend^ at 
this stage? . 



- The final step in the process is to compare the resulting sdt 
of measures to the origi^al conception of teaching. (Judgment 5).. The 
guiding question is: "Is the set of measures an ^adequate representation 
of the ^competencies consistent with the program's overall conception^ 

^ SUMMARY 



This 'model, represents one way of achieving an overview of a. 
rocess of developing an internally valid set of measures for CBTE 
programs . Having worked this through the model , consortia members 
have most likely identified many problems and thought of a few solutions 
based on their unique resources and qonstraints. The final tisk thaft 
consortia might addr^s at this conference is to begin the developjnent 
of action plans. 

* ' * 

.What action can the consortia members take (now or in the.- 
future) that might facilitate the develofDinent of CBTE assessment- pi*o- 
tedures at their institutions? Can and should resoUi'ces b^ r^IlocatQd ^ 
for the job? . . • . 



Where can consortia gain assistance with the process? 



'.What assessment-related prolJlems dp consortia need further * 
assistance with? ^ . 

If answers to thife last question aYe fortjicoming, (Consortia 
should repoVt them to the facilitator so that they can be made .public 
in th^ final general session of "the conference'. ' 



CONFERENCE EVALlfiVTION 



Alite B. Kornblith • " 

Cecile Segal 

Center for Advanced Study 'in Education of The City University o£ 

New york 



An evaluation component was included in the conf- 
erence activities to determine how successfully the program 
met the iieeds of the consortia membens. l^articipants re- 
action was collected at the. close qf^the Conference on a 
form which addressed both the suBstantive and program- 
matic aspects. Based on evaluation forms submitted by 
, approxjLmately half of the participants, the overall 
evaltiation of the Conterence was, "Good" even though it did - 
not achieve^.ar3ri"^that it set out to do. The Cbnference waa 
praised for the atmoaphere it set as well as its '''good 
organization". Partitipants liked the flexibility of the 
Conferencifc structure which permitted some adaptation to 
iridividuaA consortia needs. However, some conferees noted 
that the sessions did not get to the "nitty gritty problems 
of assessmem;;Vof providing designs which are directly 
applicable witRSnt extra ^ funding * In addition, more 
technical expertise and individual consultation wete needed. 
Another issue raised by some participants was the need for 
cl^eavi and defini^te guidelines from the State as to what 
was required for certification of their programs. 

y Part A of the evaluation form dealt with the pub- 
licly stated conference objectives. Median ratings for ^ 
each of the conference objectives can be found in Table 1. 
Each- item Vjvas rated on a 6-point scale, from Excellent to ^ . 
Not helpful at all. The Conference was successful in ^ 
making participants aware of (1) the technical, practical 
and political dimensions of developing a CBTE assessment \' 
strategy (objective 1) and (2) the potential contributions 
and concerns of public school teachers and administrators 
and" teacher education students ^d faculties (objective 5). 
Since the consortia represented were in various' stages of 
developifient of assessment systems and possessed varying 
degrees of expertise in technical matters, the median 
ratings for reaching objectives, dealing with technical 
issues were more modest. 

Part B of the evaluation form assessed the success 
of various formats for providing assistance with assessment 
problems. Figure 1 provides a summary of these data. The 
varioiis formats ^used (general sessions, workshops, role'-alike ^ 
groups, rap session) all received favorable ratings. The most 
productive of these were the role-alike groups where 785 (N°39 
of the respondents rated the i;essions as "Good" or better, 
(i.e. "Very Good" or "Excellent"). 



Table 1 

Success* In Meetijig Conference Objectives: ,Median Ratings 
Objective 




Participants will be aware o£ the . * 52 Gocxl 

technical, practical, and political ^ . 

dimensions o£ developing a CBTE ^ 

assessment strategy ' 

Participants- will ^e aware of 52 Fair 

some specific techniques that, could 
be employed to move towards assess- 
ment systems 

^Participants v/ill be aware of some 53 ^air 

6f the sources of technical assis- 
tance for developing assessment 
systems 

Participants will understand New 53 / Fair 

York State's expectations for 
assessment plans 



Participants will be aware 50 " Good 

of the potential contributions ( ^ 

and concerns of public school 

teachers and administrators 

and teacher education students 

and faculties. 

As members of consortia, par- 48 Fair 

ticipants will identify ^assess- 
ment related problems and develop 
plans for a long range assessment ' 
strategy as well as for fulfilling 
the more immediate requirements 
of a first approximation to an 
assessment system. 



*Range of ratings; Excellent, Very Good, Good, Fair, Poor, 

Not helpful at all 
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FIGURE 1 



.Si4ccess in Providing Assistance with. Assessment Probleins:. 
Percent of participants rating sessions 
as "Goocl" or better. 
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